2024-04-25T01:27:00.651 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T01:27:00.651 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329 --owner scheduled_teuthology@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/direct_io}} --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --verbose -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329/orig.config.yaml 2024-04-25T01:27:00.709 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T01:27:00.804 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/direct_io}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672329' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO+f1EuOg8WnBKeuQrsHEC4hcQAa8wWomUd2PZGxqOtVCsgzOis9uHsWodi8sdio3g4bg75WSDuj8oQ77lk51/A= smithi152.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPCGL3YxDaIsKkzZQhAdDyhLBPGvyA/tznL708dX4R3DiAn3e0Gwm4tawtLmMam7JEvt7I7N3IEJgMavOTsAYbo= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKzSTgMkewtxqV4FE/XTxe/MkrE/3A6EoC6WbQJ6/QmxPUYL78fNAC5fF52cuKoBrCNum+FhMrCejYN/d9Dw9Gw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T01:27:00.804 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-25T01:27:00.805 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-25T01:27:00.805 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T01:27:00.807 INFO:teuthology.task.internal:Checking packages... 2024-04-25T01:27:00.830 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-25T01:27:00.830 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T01:27:00.830 INFO:teuthology.packaging:ref: None 2024-04-25T01:27:00.830 INFO:teuthology.packaging:tag: None 2024-04-25T01:27:00.830 INFO:teuthology.packaging:branch: squid 2024-04-25T01:27:00.830 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:27:00.831 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-25T01:27:01.132 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-25T01:27:01.134 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T01:27:01.141 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T01:27:01.141 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T01:27:01.147 INFO:teuthology.task.internal:Saving configuration 2024-04-25T01:27:01.167 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T01:27:01.173 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T01:27:01.200 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 01:21:51.663271', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO+f1EuOg8WnBKeuQrsHEC4hcQAa8wWomUd2PZGxqOtVCsgzOis9uHsWodi8sdio3g4bg75WSDuj8oQ77lk51/A='} 2024-04-25T01:27:01.223 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 01:21:51.662254', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPCGL3YxDaIsKkzZQhAdDyhLBPGvyA/tznL708dX4R3DiAn3e0Gwm4tawtLmMam7JEvt7I7N3IEJgMavOTsAYbo='} 2024-04-25T01:27:01.249 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 01:21:51.661001', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKzSTgMkewtxqV4FE/XTxe/MkrE/3A6EoC6WbQJ6/QmxPUYL78fNAC5fF52cuKoBrCNum+FhMrCejYN/d9Dw9Gw='} 2024-04-25T01:27:01.249 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T01:27:01.256 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T01:27:01.256 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T01:27:01.257 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T01:27:01.257 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T01:27:01.362 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fccdf5f6160>, signals=[15]) 2024-04-25T01:27:01.363 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T01:27:01.373 INFO:teuthology.task.internal:Opening connections... 2024-04-25T01:27:01.373 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T01:27:01.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:27:01.458 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi152.front.sepia.ceph.com 2024-04-25T01:27:01.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:27:01.543 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T01:27:01.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:27:01.623 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T01:27:01.632 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-04-25T01:27:01.649 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-04-25T01:27:01.650 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-04-25T01:27:01.707 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream" 2024-04-25T01:27:01.707 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="9" 2024-04-25T01:27:01.707 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos" 2024-04-25T01:27:01.707 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora" 2024-04-25T01:27:01.708 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="9" 2024-04-25T01:27:01.708 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el9" 2024-04-25T01:27:01.708 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T01:27:01.708 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-04-25T01:27:01.708 INFO:teuthology.orchestra.run.smithi078.stdout:LOGO="fedora-logo-icon" 2024-04-25T01:27:01.708 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T01:27:01.708 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/" 2024-04-25T01:27:01.708 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T01:27:01.708 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T01:27:01.708 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T01:27:01.709 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-04-25T01:27:01.738 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2024-04-25T01:27:01.756 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2024-04-25T01:27:01.756 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2024-04-25T01:27:01.814 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="CentOS Stream" 2024-04-25T01:27:01.814 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="9" 2024-04-25T01:27:01.814 INFO:teuthology.orchestra.run.smithi152.stdout:ID="centos" 2024-04-25T01:27:01.814 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="rhel fedora" 2024-04-25T01:27:01.814 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="9" 2024-04-25T01:27:01.815 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el9" 2024-04-25T01:27:01.815 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T01:27:01.815 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2024-04-25T01:27:01.815 INFO:teuthology.orchestra.run.smithi152.stdout:LOGO="fedora-logo-icon" 2024-04-25T01:27:01.815 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T01:27:01.815 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://centos.org/" 2024-04-25T01:27:01.815 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T01:27:01.815 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T01:27:01.815 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T01:27:01.815 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2024-04-25T01:27:01.838 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-04-25T01:27:01.856 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-04-25T01:27:01.857 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-04-25T01:27:01.914 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-04-25T01:27:01.914 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-04-25T01:27:01.914 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-04-25T01:27:01.914 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-04-25T01:27:01.914 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-04-25T01:27:01.914 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-04-25T01:27:01.914 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T01:27:01.914 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-04-25T01:27:01.914 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-04-25T01:27:01.914 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T01:27:01.914 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-04-25T01:27:01.915 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T01:27:01.915 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T01:27:01.915 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T01:27:01.915 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-04-25T01:27:01.938 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T01:27:01.949 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T01:27:01.955 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T01:27:01.955 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2024-04-25T01:27:01.958 DEBUG:teuthology.orchestra.run.smithi152:> test '!' -e /home/ubuntu/cephtest 2024-04-25T01:27:01.960 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-04-25T01:27:01.976 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T01:27:01.982 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T01:27:01.983 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2024-04-25T01:27:02.015 DEBUG:teuthology.orchestra.run.smithi152:> test -z $(ls -A /var/lib/ceph) 2024-04-25T01:27:02.018 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-04-25T01:27:02.045 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T01:27:02.115 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T01:27:02.129 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-25T01:27:02.130 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T01:27:02.130 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T01:27:02.130 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T01:27:02.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:27:02.146 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-04-25T01:27:02.204 INFO:teuthology.orchestra.run.smithi078.stdout:5.14.0-437.el9.x86_64 2024-04-25T01:27:02.204 INFO:teuthology.task.kernel:Running kernel on smithi078: 5.14.0-437.el9.x86_64 2024-04-25T01:27:02.204 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum install -y kernel 2024-04-25T01:27:04.505 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T01:27:19.787 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-04-25T01:27:24.196 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.2 MB 00:00 2024-04-25T01:27:25.814 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-04-25T01:27:27.699 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-25T01:27:32.657 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-04-25T01:27:34.538 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T01:27:34.538 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T01:27:34.625 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-25T01:27:34.626 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-25T01:27:34.626 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-25T01:27:34.736 DEBUG:teuthology.orchestra.run.smithi078:> echo no | sudo yum reinstall kernel || true 2024-04-25T01:27:35.233 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 01:27:32 AM UTC. 2024-04-25T01:27:35.352 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-25T01:27:35.365 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-25T01:27:35.365 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-04-25T01:27:35.365 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-25T01:27:35.365 INFO:teuthology.orchestra.run.smithi078.stdout:Reinstalling: 2024-04-25T01:27:35.365 INFO:teuthology.orchestra.run.smithi078.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T01:27:35.365 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:27:35.365 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-25T01:27:35.365 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-25T01:27:35.365 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:27:35.365 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 255 k 2024-04-25T01:27:35.365 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 0 2024-04-25T01:27:35.366 INFO:teuthology.orchestra.run.smithi078.stderr:Operation aborted. 2024-04-25T01:27:35.422 INFO:teuthology.orchestra.run.smithi078.stdout:Is this ok [y/N]: 2024-04-25T01:27:35.422 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum reinstall -y kernel || true 2024-04-25T01:27:35.913 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 01:27:32 AM UTC. 2024-04-25T01:27:36.028 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-25T01:27:36.029 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-25T01:27:36.029 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-04-25T01:27:36.029 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-25T01:27:36.030 INFO:teuthology.orchestra.run.smithi078.stdout:Reinstalling: 2024-04-25T01:27:36.030 INFO:teuthology.orchestra.run.smithi078.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T01:27:36.030 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:27:36.030 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-25T01:27:36.030 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-25T01:27:36.030 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:27:36.030 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 255 k 2024-04-25T01:27:36.030 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 0 2024-04-25T01:27:36.030 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-25T01:27:36.451 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.7 MB/s | 255 kB 00:00 2024-04-25T01:27:36.451 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:27:36.451 INFO:teuthology.orchestra.run.smithi078.stdout:Total 605 kB/s | 255 kB 00:00 2024-04-25T01:27:36.473 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-25T01:27:36.660 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-25T01:27:36.660 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-25T01:27:36.665 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-25T01:27:36.665 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-25T01:27:36.805 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-25T01:27:36.877 INFO:teuthology.orchestra.run.smithi078.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T01:27:36.877 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:27:37.580 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:27:37.580 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T01:27:38.270 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:27:38.271 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:27:38.271 INFO:teuthology.orchestra.run.smithi078.stdout:Reinstalled: 2024-04-25T01:27:38.271 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T01:27:38.271 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:27:38.271 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-25T01:27:38.438 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T01:27:38.484 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T01:27:38.485 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T01:27:38.485 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T01:27:38.485 DEBUG:teuthology.parallel:result is None 2024-04-25T01:27:38.485 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T01:27:38.492 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T01:27:38.492 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T01:27:38.494 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T01:27:38.497 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T01:27:38.515 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T01:27:38.521 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T01:27:38.527 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T01:27:38.527 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T01:27:38.549 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T01:27:38.555 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T01:27:38.653 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T01:27:38.660 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T01:27:38.660 DEBUG:teuthology.orchestra.run.smithi152:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T01:27:38.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:27:38.677 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T01:27:38.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:27:38.694 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T01:27:38.696 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T01:27:38.720 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T01:27:38.721 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T01:27:38.733 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T01:27:38.752 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T01:27:38.766 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T01:27:38.783 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T01:27:38.788 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T01:27:38.790 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T01:27:38.797 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T01:27:38.797 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T01:27:38.800 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T01:27:38.833 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T01:27:38.859 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T01:27:38.868 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T01:27:38.869 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T01:27:38.871 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T01:27:38.900 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T01:27:38.915 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T01:27:38.963 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T01:27:39.011 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T01:27:39.092 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T01:27:39.129 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:27:39.130 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T01:27:39.192 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T01:27:39.235 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T01:27:39.285 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T01:27:39.368 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T01:27:39.408 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T01:27:39.408 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T01:27:39.472 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T01:27:39.516 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T01:27:39.562 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T01:27:39.644 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T01:27:39.688 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T01:27:39.688 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T01:27:39.754 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2024-04-25T01:27:39.757 DEBUG:teuthology.orchestra.run.smithi152:> sudo service rsyslog restart 2024-04-25T01:27:39.759 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-04-25T01:27:39.788 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:27:39.795 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:27:39.846 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:27:40.278 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T01:27:40.285 INFO:teuthology.task.internal:Starting timer... 2024-04-25T01:27:40.285 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T01:27:40.313 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T01:27:40.321 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T01:27:40.322 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2024-04-25T01:27:40.368 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2024-04-25T01:27:40.370 DEBUG:teuthology.orchestra.run.smithi152:> sudo service auditd rotate 2024-04-25T01:27:40.434 INFO:teuthology.orchestra.run.smithi152.stdout:Rotating logs: 2024-04-25T01:27:40.436 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-04-25T01:27:40.509 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-04-25T01:27:40.512 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T01:27:40.512 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2024-04-25T01:27:40.533 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2024-04-25T01:27:40.534 DEBUG:teuthology.orchestra.run.smithi152:> /usr/sbin/getenforce 2024-04-25T01:27:40.555 INFO:teuthology.orchestra.run.smithi152.stdout:Permissive 2024-04-25T01:27:40.555 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-04-25T01:27:40.576 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-04-25T01:27:40.576 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi078.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi152.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-04-25T01:27:40.576 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:27:40.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:27:40.602 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:27:40.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:27:40.629 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:27:40.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:27:40.657 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T01:27:40.657 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2024-04-25T01:27:40.683 DEBUG:teuthology.orchestra.run.smithi152:> sudo /usr/sbin/setenforce permissive 2024-04-25T01:27:40.740 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-04-25T01:27:40.766 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T01:27:40.775 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T01:27:40.780 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T01:27:40.911 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T01:27:40.927 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T01:27:40.928 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi078.front.sepia.ceph.com,smithi152.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T01:31:45.035 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi078.front.sepia.ceph.com'), Remote(name='ubuntu@smithi152.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-04-25T01:31:45.036 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-04-25T01:31:45.037 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:31:45.113 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-04-25T01:31:45.194 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-04-25T01:31:45.194 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-04-25T01:31:45.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:31:45.269 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-04-25T01:31:45.347 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-04-25T01:31:45.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T01:31:45.348 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:31:45.424 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-04-25T01:31:45.506 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T01:31:45.506 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T01:31:45.516 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T01:31:45.516 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T01:31:45.517 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:31:45.520 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T01:31:45.520 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:31:45.522 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T01:31:45.522 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:31:45.549 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T01:31:45.550 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T01:31:45.564 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T01:31:45.565 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T01:31:45.574 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T01:31:45.588 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: ntpd: command not found 2024-04-25T01:31:45.588 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T01:31:45.591 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2024-04-25T01:31:45.598 INFO:teuthology.orchestra.run.smithi152.stdout:506 Cannot talk to daemon 2024-04-25T01:31:45.603 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2024-04-25T01:31:45.612 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T01:31:45.612 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-04-25T01:31:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T01:31:45.623 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-04-25T01:31:45.625 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T01:31:45.634 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T01:31:45.637 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T01:31:45.650 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T01:31:45.680 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:31:45.682 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:31:45.682 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-04-25T01:31:45.683 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:45.683 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:45.683 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:45.683 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:45.700 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:31:45.703 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:31:45.703 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-04-25T01:31:45.704 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:45.704 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:45.704 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:45.704 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:45.704 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:31:45.707 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:31:45.707 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-04-25T01:31:45.707 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:45.707 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:45.707 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:45.707 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:31:45.708 INFO:teuthology.run_tasks:Running task install... 2024-04-25T01:31:45.717 DEBUG:teuthology.task.install:project ceph 2024-04-25T01:31:45.717 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T01:31:45.717 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T01:31:45.717 INFO:teuthology.task.install:Using flavor: default 2024-04-25T01:31:45.725 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T01:31:45.725 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T01:31:45.725 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T01:31:45.726 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:31:45.728 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T01:31:45.728 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:31:45.729 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T01:31:45.729 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:31:45.880 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T01:31:45.881 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T01:31:45.991 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T01:31:45.991 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T01:31:45.991 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T01:31:46.015 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T01:31:46.015 DEBUG:teuthology.orchestra.run.smithi152:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T01:31:46.037 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T01:31:46.037 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T01:31:46.043 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T01:31:46.043 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T01:31:46.093 DEBUG:teuthology.orchestra.run.smithi152:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T01:31:46.166 DEBUG:teuthology.orchestra.run.smithi152:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T01:31:46.210 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T01:31:46.210 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:31:46.210 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T01:31:46.218 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T01:31:46.218 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T01:31:46.218 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T01:31:46.229 INFO:teuthology.orchestra.run.smithi152.stdout:check_obsoletes = 1 2024-04-25T01:31:46.231 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum clean all 2024-04-25T01:31:46.239 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T01:31:46.239 DEBUG:teuthology.orchestra.run.smithi078:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T01:31:46.243 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T01:31:46.243 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T01:31:46.307 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T01:31:46.308 DEBUG:teuthology.orchestra.run.smithi078:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T01:31:46.379 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T01:31:46.388 DEBUG:teuthology.orchestra.run.smithi078:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T01:31:46.442 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-04-25T01:31:46.444 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-04-25T01:31:46.456 INFO:teuthology.orchestra.run.smithi078.stdout:check_obsoletes = 1 2024-04-25T01:31:46.456 INFO:teuthology.orchestra.run.smithi152.stdout:48 files removed 2024-04-25T01:31:46.458 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum clean all 2024-04-25T01:31:46.484 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T01:31:46.671 INFO:teuthology.orchestra.run.smithi196.stdout:48 files removed 2024-04-25T01:31:46.699 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T01:31:46.701 INFO:teuthology.orchestra.run.smithi078.stdout:48 files removed 2024-04-25T01:31:46.728 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T01:31:47.169 INFO:teuthology.orchestra.run.smithi152.stdout:ceph packages for x86_64 196 kB/s | 83 kB 00:00 2024-04-25T01:31:47.380 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 197 kB/s | 83 kB 00:00 2024-04-25T01:31:47.391 INFO:teuthology.orchestra.run.smithi078.stdout:ceph packages for x86_64 206 kB/s | 83 kB 00:00 2024-04-25T01:31:47.619 INFO:teuthology.orchestra.run.smithi152.stdout:ceph noarch packages 28 kB/s | 11 kB 00:00 2024-04-25T01:31:47.751 INFO:teuthology.orchestra.run.smithi078.stdout:ceph noarch packages 36 kB/s | 11 kB 00:00 2024-04-25T01:31:47.823 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-04-25T01:31:47.987 INFO:teuthology.orchestra.run.smithi152.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-25T01:31:48.156 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 6.3 kB/s | 1.9 kB 00:00 2024-04-25T01:31:48.236 INFO:teuthology.orchestra.run.smithi078.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-04-25T01:31:48.595 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-25T01:31:49.409 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 7.1 MB/s | 8.0 MB 00:01 2024-04-25T01:31:49.690 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.0 MB 00:01 2024-04-25T01:31:50.866 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-25T01:31:51.583 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T01:31:55.647 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.2 MB 00:01 2024-04-25T01:31:56.142 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 6.2 MB 00:00 2024-04-25T01:31:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-25T01:31:57.902 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-25T01:31:59.127 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-25T01:32:04.193 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-25T01:32:06.042 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-04-25T01:32:06.310 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:32:06.311 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:32:06.321 INFO:teuthology.orchestra.run.smithi196.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T01:32:06.322 INFO:teuthology.orchestra.run.smithi196.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T01:32:06.323 INFO:teuthology.orchestra.run.smithi196.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T01:32:06.324 INFO:teuthology.orchestra.run.smithi196.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T01:32:06.325 INFO:teuthology.orchestra.run.smithi196.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T01:32:06.327 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T01:32:06.328 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T01:32:06.388 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-04-25T01:32:06.411 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-04-25T01:32:06.411 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-04-25T01:32:06.411 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-04-25T01:32:06.411 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-04-25T01:32:06.411 INFO:teuthology.orchestra.run.smithi196.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T01:32:06.411 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T01:32:06.411 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T01:32:06.411 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T01:32:06.411 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T01:32:06.412 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T01:32:06.412 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T01:32:06.412 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T01:32:06.412 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T01:32:06.412 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T01:32:06.412 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T01:32:06.412 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T01:32:06.412 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T01:32:06.412 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T01:32:06.412 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T01:32:06.412 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T01:32:06.412 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T01:32:06.412 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T01:32:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T01:32:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T01:32:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T01:32:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T01:32:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T01:32:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T01:32:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T01:32:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T01:32:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T01:32:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T01:32:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T01:32:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T01:32:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T01:32:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T01:32:06.414 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T01:32:06.414 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T01:32:06.414 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:32:06.414 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T01:32:06.414 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T01:32:06.414 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-04-25T01:32:06.414 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T01:32:06.414 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:32:06.414 INFO:teuthology.orchestra.run.smithi196.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T01:32:06.414 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T01:32:06.414 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-04-25T01:32:06.414 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T01:32:06.415 INFO:teuthology.orchestra.run.smithi196.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T01:32:06.415 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T01:32:06.415 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T01:32:06.415 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T01:32:06.415 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T01:32:06.415 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T01:32:06.415 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T01:32:06.415 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T01:32:06.415 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T01:32:06.415 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T01:32:06.415 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T01:32:06.415 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T01:32:06.415 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T01:32:06.415 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T01:32:06.416 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T01:32:06.416 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T01:32:06.416 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T01:32:06.416 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T01:32:06.416 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T01:32:06.416 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T01:32:06.416 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T01:32:06.416 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T01:32:06.416 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T01:32:06.416 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T01:32:06.416 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T01:32:06.416 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T01:32:06.416 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T01:32:06.416 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T01:32:06.417 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T01:32:06.417 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T01:32:06.417 INFO:teuthology.orchestra.run.smithi196.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T01:32:06.417 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T01:32:06.417 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T01:32:06.417 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T01:32:06.417 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T01:32:06.417 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T01:32:06.417 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T01:32:06.417 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T01:32:06.417 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T01:32:06.417 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T01:32:06.417 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T01:32:06.417 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T01:32:06.418 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T01:32:06.418 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T01:32:06.418 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T01:32:06.418 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T01:32:06.418 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T01:32:06.418 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T01:32:06.418 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T01:32:06.418 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T01:32:06.418 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T01:32:06.418 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T01:32:06.418 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T01:32:06.418 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T01:32:06.418 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T01:32:06.419 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T01:32:06.419 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T01:32:06.419 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T01:32:06.419 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T01:32:06.419 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T01:32:06.419 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T01:32:06.419 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T01:32:06.419 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T01:32:06.419 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T01:32:06.419 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T01:32:06.419 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T01:32:06.419 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T01:32:06.419 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T01:32:06.419 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T01:32:06.420 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T01:32:06.420 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T01:32:06.420 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T01:32:06.420 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T01:32:06.420 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T01:32:06.420 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T01:32:06.420 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T01:32:06.420 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T01:32:06.420 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T01:32:06.420 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T01:32:06.420 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T01:32:06.420 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T01:32:06.420 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T01:32:06.420 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T01:32:06.421 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T01:32:06.421 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T01:32:06.421 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T01:32:06.421 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T01:32:06.421 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T01:32:06.421 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T01:32:06.421 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T01:32:06.421 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T01:32:06.421 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T01:32:06.421 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T01:32:06.421 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T01:32:06.421 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T01:32:06.421 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T01:32:06.421 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T01:32:06.422 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T01:32:06.422 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T01:32:06.422 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T01:32:06.422 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T01:32:06.422 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T01:32:06.422 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T01:32:06.422 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T01:32:06.422 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T01:32:06.422 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T01:32:06.422 INFO:teuthology.orchestra.run.smithi196.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T01:32:06.422 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T01:32:06.422 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T01:32:06.422 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-04-25T01:32:06.422 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T01:32:06.423 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:32:06.423 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-04-25T01:32:06.423 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-04-25T01:32:06.423 INFO:teuthology.orchestra.run.smithi196.stdout:Install 148 Packages 2024-04-25T01:32:06.423 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 4 Packages 2024-04-25T01:32:06.423 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:32:06.423 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 220 M 2024-04-25T01:32:06.424 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-04-25T01:32:07.788 INFO:teuthology.orchestra.run.smithi196.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-04-25T01:32:08.046 INFO:teuthology.orchestra.run.smithi196.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 3.4 MB/s | 890 kB 00:00 2024-04-25T01:32:08.096 INFO:teuthology.orchestra.run.smithi196.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 2.8 MB/s | 145 kB 00:00 2024-04-25T01:32:08.397 INFO:teuthology.orchestra.run.smithi196.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 7.9 MB/s | 2.4 MB 00:00 2024-04-25T01:32:08.580 INFO:teuthology.orchestra.run.smithi196.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 8.2 MB/s | 1.5 MB 00:00 2024-04-25T01:32:09.081 INFO:teuthology.orchestra.run.smithi196.stdout:(6/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 3.1 MB/s | 5.4 MB 00:01 2024-04-25T01:32:09.147 INFO:teuthology.orchestra.run.smithi196.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 8.3 MB/s | 4.7 MB 00:00 2024-04-25T01:32:09.748 INFO:teuthology.orchestra.run.smithi196.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 8.7 MB/s | 21 MB 00:02 2024-04-25T01:32:09.806 INFO:teuthology.orchestra.run.smithi196.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 433 kB/s | 25 kB 00:00 2024-04-25T01:32:10.107 INFO:teuthology.orchestra.run.smithi196.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 11 MB/s | 10 MB 00:00 2024-04-25T01:32:10.110 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-25T01:32:10.190 INFO:teuthology.orchestra.run.smithi196.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.6 MB/s | 222 kB 00:00 2024-04-25T01:32:10.282 INFO:teuthology.orchestra.run.smithi196.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 296 kB/s | 27 kB 00:00 2024-04-25T01:32:10.357 INFO:teuthology.orchestra.run.smithi196.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 423 kB/s | 32 kB 00:00 2024-04-25T01:32:11.032 INFO:teuthology.orchestra.run.smithi196.stdout:(14/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 8.6 MB/s | 17 MB 00:01 2024-04-25T01:32:11.058 INFO:teuthology.orchestra.run.smithi196.stdout:(15/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 1.0 MB/s | 745 kB 00:00 2024-04-25T01:32:11.108 INFO:teuthology.orchestra.run.smithi196.stdout:(16/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.1 MB/s | 163 kB 00:00 2024-04-25T01:32:11.133 INFO:teuthology.orchestra.run.smithi196.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 1.6 MB/s | 127 kB 00:00 2024-04-25T01:32:11.200 INFO:teuthology.orchestra.run.smithi196.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 5.3 MB/s | 501 kB 00:00 2024-04-25T01:32:11.292 INFO:teuthology.orchestra.run.smithi196.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 492 kB/s | 45 kB 00:00 2024-04-25T01:32:11.401 INFO:teuthology.orchestra.run.smithi196.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.2 MB/s | 133 kB 00:00 2024-04-25T01:32:11.493 INFO:teuthology.orchestra.run.smithi196.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.7 MB/s | 162 kB 00:00 2024-04-25T01:32:11.585 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-04-25T01:32:11.593 INFO:teuthology.orchestra.run.smithi196.stdout:(22/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 12 MB/s | 5.3 MB 00:00 2024-04-25T01:32:11.618 INFO:teuthology.orchestra.run.smithi196.stdout:(23/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.5 MB/s | 321 kB 00:00 2024-04-25T01:32:11.660 INFO:teuthology.orchestra.run.smithi196.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 4.4 MB/s | 299 kB 00:00 2024-04-25T01:32:11.694 INFO:teuthology.orchestra.run.smithi196.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.3 MB/s | 99 kB 00:00 2024-04-25T01:32:11.719 INFO:teuthology.orchestra.run.smithi196.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-25T01:32:11.819 INFO:teuthology.orchestra.run.smithi196.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.7 MB/s | 171 kB 00:00 2024-04-25T01:32:11.886 INFO:teuthology.orchestra.run.smithi196.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 476 kB/s | 32 kB 00:00 2024-04-25T01:32:11.962 INFO:teuthology.orchestra.run.smithi196.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.8 MB/s | 141 kB 00:00 2024-04-25T01:32:12.062 INFO:teuthology.orchestra.run.smithi196.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 8.4 MB/s | 3.1 MB 00:00 2024-04-25T01:32:12.363 INFO:teuthology.orchestra.run.smithi196.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 8.8 MB/s | 3.5 MB 00:00 2024-04-25T01:32:12.438 INFO:teuthology.orchestra.run.smithi196.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 3.3 MB/s | 249 kB 00:00 2024-04-25T01:32:12.504 INFO:teuthology.orchestra.run.smithi196.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 744 kB/s | 49 kB 00:00 2024-04-25T01:32:12.602 INFO:teuthology.orchestra.run.smithi196.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 161 kB/s | 15 kB 00:00 2024-04-25T01:32:12.680 INFO:teuthology.orchestra.run.smithi196.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.2 MB/s | 264 kB 00:00 2024-04-25T01:32:12.704 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-04-25T01:32:12.839 INFO:teuthology.orchestra.run.smithi196.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 4.7 MB/s | 762 kB 00:00 2024-04-25T01:32:12.939 INFO:teuthology.orchestra.run.smithi196.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 8.4 MB/s | 7.4 MB 00:00 2024-04-25T01:32:12.964 INFO:teuthology.orchestra.run.smithi196.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 146 kB/s | 18 kB 00:00 2024-04-25T01:32:13.390 INFO:teuthology.orchestra.run.smithi196.stdout:(39/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 13 MB/s | 48 MB 00:03 2024-04-25T01:32:13.540 INFO:teuthology.orchestra.run.smithi196.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 134 kB/s | 80 kB 00:00 2024-04-25T01:32:13.690 INFO:teuthology.orchestra.run.smithi196.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 651 kB/s | 195 kB 00:00 2024-04-25T01:32:13.715 INFO:teuthology.orchestra.run.smithi196.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-04-25T01:32:13.790 INFO:teuthology.orchestra.run.smithi196.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-04-25T01:32:13.924 INFO:teuthology.orchestra.run.smithi196.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 511 kB/s | 106 kB 00:00 2024-04-25T01:32:13.999 INFO:teuthology.orchestra.run.smithi196.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 649 kB/s | 135 kB 00:00 2024-04-25T01:32:14.108 INFO:teuthology.orchestra.run.smithi196.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 2.9 MB/s | 1.2 MB 00:00 2024-04-25T01:32:14.133 INFO:teuthology.orchestra.run.smithi196.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 598 kB/s | 125 kB 00:00 2024-04-25T01:32:14.250 INFO:teuthology.orchestra.run.smithi196.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 857 kB/s | 215 kB 00:00 2024-04-25T01:32:14.450 INFO:teuthology.orchestra.run.smithi196.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 689 kB 00:00 2024-04-25T01:32:14.475 INFO:teuthology.orchestra.run.smithi196.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 474 kB/s | 107 kB 00:00 2024-04-25T01:32:14.509 INFO:teuthology.orchestra.run.smithi196.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 943 kB 00:00 2024-04-25T01:32:14.626 INFO:teuthology.orchestra.run.smithi196.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 123 kB/s | 21 kB 00:00 2024-04-25T01:32:14.652 INFO:teuthology.orchestra.run.smithi196.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 231 kB/s | 33 kB 00:00 2024-04-25T01:32:14.726 INFO:teuthology.orchestra.run.smithi196.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 1.2 MB/s | 313 kB 00:00 2024-04-25T01:32:14.785 INFO:teuthology.orchestra.run.smithi196.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 136 kB/s | 18 kB 00:00 2024-04-25T01:32:14.852 INFO:teuthology.orchestra.run.smithi196.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 89 kB/s | 11 kB 00:00 2024-04-25T01:32:15.002 INFO:teuthology.orchestra.run.smithi196.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 739 kB/s | 160 kB 00:00 2024-04-25T01:32:15.027 INFO:teuthology.orchestra.run.smithi196.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 258 kB/s | 45 kB 00:00 2024-04-25T01:32:15.336 INFO:teuthology.orchestra.run.smithi196.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-04-25T01:32:15.386 INFO:teuthology.orchestra.run.smithi196.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.9 MB/s | 3.0 MB 00:00 2024-04-25T01:32:15.420 INFO:teuthology.orchestra.run.smithi196.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 1.4 MB/s | 582 kB 00:00 2024-04-25T01:32:15.495 INFO:teuthology.orchestra.run.smithi196.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 160 kB/s | 25 kB 00:00 2024-04-25T01:32:15.579 INFO:teuthology.orchestra.run.smithi196.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 247 kB/s | 47 kB 00:00 2024-04-25T01:32:15.671 INFO:teuthology.orchestra.run.smithi196.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.1 MB/s | 292 kB 00:00 2024-04-25T01:32:15.721 INFO:teuthology.orchestra.run.smithi196.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 248 kB/s | 35 kB 00:00 2024-04-25T01:32:15.788 INFO:teuthology.orchestra.run.smithi196.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 644 kB/s | 188 kB 00:00 2024-04-25T01:32:16.078 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:18 2024-04-25T01:32:16.196 INFO:teuthology.orchestra.run.smithi196.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-04-25T01:32:16.397 INFO:teuthology.orchestra.run.smithi196.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.5 MB/s | 4.7 MB 00:00 2024-04-25T01:32:16.580 INFO:teuthology.orchestra.run.smithi196.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 2.3 MB/s | 911 kB 00:00 2024-04-25T01:32:16.647 INFO:teuthology.orchestra.run.smithi196.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.8 MB/s | 4.4 MB 00:00 2024-04-25T01:32:16.672 INFO:teuthology.orchestra.run.smithi196.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 484 kB/s | 133 kB 00:00 2024-04-25T01:32:16.873 INFO:teuthology.orchestra.run.smithi196.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 437 kB/s | 87 kB 00:00 2024-04-25T01:32:17.048 INFO:teuthology.orchestra.run.smithi196.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.5 MB/s | 1.0 MB 00:00 2024-04-25T01:32:17.348 INFO:teuthology.orchestra.run.smithi196.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 813 kB/s | 244 kB 00:00 2024-04-25T01:32:17.598 INFO:teuthology.orchestra.run.smithi196.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 992 kB/s | 248 kB 00:00 2024-04-25T01:32:17.765 INFO:teuthology.orchestra.run.smithi196.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 154 kB/s | 26 kB 00:00 2024-04-25T01:32:17.832 INFO:teuthology.orchestra.run.smithi196.stdout:(77/152): postgresql-server-13.14-1.el9.x86_64. 4.6 MB/s | 5.8 MB 00:01 2024-04-25T01:32:17.941 INFO:teuthology.orchestra.run.smithi196.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 109 kB/s | 19 kB 00:00 2024-04-25T01:32:18.038 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 6.1 kB/s | 1.7 kB 00:00 2024-04-25T01:32:18.091 INFO:teuthology.orchestra.run.smithi196.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 666 kB/s | 172 kB 00:00 2024-04-25T01:32:18.116 INFO:teuthology.orchestra.run.smithi196.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 198 kB/s | 35 kB 00:00 2024-04-25T01:32:18.366 INFO:teuthology.orchestra.run.smithi196.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 250 kB 00:00 2024-04-25T01:32:18.667 INFO:teuthology.orchestra.run.smithi196.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 741 kB/s | 222 kB 00:00 2024-04-25T01:32:18.775 INFO:teuthology.orchestra.run.smithi196.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 3.1 MB/s | 6.0 MB 00:01 2024-04-25T01:32:18.842 INFO:teuthology.orchestra.run.smithi196.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.8 MB/s | 5.1 MB 00:00 2024-04-25T01:32:18.876 INFO:teuthology.orchestra.run.smithi196.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 369 kB/s | 77 kB 00:00 2024-04-25T01:32:18.993 INFO:teuthology.orchestra.run.smithi196.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 195 kB/s | 42 kB 00:00 2024-04-25T01:32:19.135 INFO:teuthology.orchestra.run.smithi196.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 922 kB/s | 269 kB 00:00 2024-04-25T01:32:19.160 INFO:teuthology.orchestra.run.smithi196.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 559 kB/s | 159 kB 00:00 2024-04-25T01:32:19.285 INFO:teuthology.orchestra.run.smithi196.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 954 kB/s | 279 kB 00:00 2024-04-25T01:32:19.310 INFO:teuthology.orchestra.run.smithi196.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 291 kB/s | 51 kB 00:00 2024-04-25T01:32:19.360 INFO:teuthology.orchestra.run.smithi196.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 268 kB/s | 54 kB 00:00 2024-04-25T01:32:19.503 INFO:teuthology.orchestra.run.smithi196.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 217 kB/s | 42 kB 00:00 2024-04-25T01:32:19.536 INFO:teuthology.orchestra.run.smithi196.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 251 kB/s | 44 kB 00:00 2024-04-25T01:32:19.695 INFO:teuthology.orchestra.run.smithi196.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 227 kB/s | 36 kB 00:00 2024-04-25T01:32:19.778 INFO:teuthology.orchestra.run.smithi196.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 801 kB/s | 221 kB 00:00 2024-04-25T01:32:19.920 INFO:teuthology.orchestra.run.smithi196.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 210 kB/s | 47 kB 00:00 2024-04-25T01:32:20.145 INFO:teuthology.orchestra.run.smithi196.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 831 kB/s | 305 kB 00:00 2024-04-25T01:32:20.147 INFO:teuthology.orchestra.run.smithi078.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:32:20.147 INFO:teuthology.orchestra.run.smithi078.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:32:20.158 INFO:teuthology.orchestra.run.smithi078.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T01:32:20.158 INFO:teuthology.orchestra.run.smithi078.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T01:32:20.159 INFO:teuthology.orchestra.run.smithi078.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T01:32:20.160 INFO:teuthology.orchestra.run.smithi078.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T01:32:20.162 INFO:teuthology.orchestra.run.smithi078.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T01:32:20.164 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T01:32:20.165 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T01:32:20.171 INFO:teuthology.orchestra.run.smithi196.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 231 kB/s | 58 kB 00:00 2024-04-25T01:32:20.224 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-25T01:32:20.233 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-04-25T01:32:20.233 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-25T01:32:20.233 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-04-25T01:32:20.233 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-25T01:32:20.233 INFO:teuthology.orchestra.run.smithi078.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T01:32:20.233 INFO:teuthology.orchestra.run.smithi078.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T01:32:20.233 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T01:32:20.233 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T01:32:20.233 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T01:32:20.234 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T01:32:20.234 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T01:32:20.234 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T01:32:20.234 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T01:32:20.234 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T01:32:20.234 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T01:32:20.234 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T01:32:20.234 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T01:32:20.234 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T01:32:20.234 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T01:32:20.234 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T01:32:20.234 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T01:32:20.234 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T01:32:20.235 INFO:teuthology.orchestra.run.smithi078.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T01:32:20.235 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T01:32:20.235 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T01:32:20.235 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T01:32:20.235 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T01:32:20.235 INFO:teuthology.orchestra.run.smithi078.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T01:32:20.235 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T01:32:20.235 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T01:32:20.235 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T01:32:20.235 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T01:32:20.235 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T01:32:20.235 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T01:32:20.235 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T01:32:20.236 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T01:32:20.236 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T01:32:20.236 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T01:32:20.236 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:32:20.236 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T01:32:20.236 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T01:32:20.236 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrading: 2024-04-25T01:32:20.236 INFO:teuthology.orchestra.run.smithi078.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T01:32:20.236 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:32:20.236 INFO:teuthology.orchestra.run.smithi078.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T01:32:20.236 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T01:32:20.236 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-25T01:32:20.236 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T01:32:20.237 INFO:teuthology.orchestra.run.smithi078.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T01:32:20.237 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T01:32:20.237 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T01:32:20.237 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T01:32:20.237 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T01:32:20.237 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T01:32:20.237 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T01:32:20.237 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T01:32:20.237 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T01:32:20.238 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T01:32:20.238 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T01:32:20.238 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T01:32:20.238 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T01:32:20.238 INFO:teuthology.orchestra.run.smithi078.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T01:32:20.238 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T01:32:20.238 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T01:32:20.238 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T01:32:20.238 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T01:32:20.239 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T01:32:20.239 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T01:32:20.239 INFO:teuthology.orchestra.run.smithi078.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T01:32:20.239 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T01:32:20.239 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T01:32:20.239 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T01:32:20.239 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T01:32:20.239 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T01:32:20.239 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T01:32:20.239 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T01:32:20.239 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T01:32:20.239 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T01:32:20.239 INFO:teuthology.orchestra.run.smithi078.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T01:32:20.239 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T01:32:20.240 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T01:32:20.240 INFO:teuthology.orchestra.run.smithi078.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T01:32:20.240 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T01:32:20.240 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T01:32:20.240 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T01:32:20.240 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T01:32:20.240 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T01:32:20.240 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T01:32:20.240 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T01:32:20.240 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T01:32:20.240 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T01:32:20.240 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T01:32:20.240 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T01:32:20.241 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T01:32:20.242 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T01:32:20.242 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T01:32:20.242 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T01:32:20.242 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T01:32:20.242 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T01:32:20.242 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T01:32:20.242 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T01:32:20.243 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T01:32:20.243 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T01:32:20.243 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T01:32:20.243 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T01:32:20.243 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T01:32:20.243 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T01:32:20.243 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T01:32:20.243 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T01:32:20.243 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T01:32:20.244 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T01:32:20.244 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T01:32:20.244 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T01:32:20.244 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T01:32:20.245 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T01:32:20.245 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T01:32:20.245 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T01:32:20.245 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T01:32:20.245 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T01:32:20.245 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T01:32:20.245 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T01:32:20.245 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T01:32:20.245 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T01:32:20.245 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T01:32:20.245 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T01:32:20.245 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T01:32:20.245 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T01:32:20.246 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T01:32:20.246 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T01:32:20.246 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T01:32:20.246 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T01:32:20.246 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T01:32:20.246 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T01:32:20.246 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T01:32:20.246 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T01:32:20.246 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T01:32:20.246 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T01:32:20.246 INFO:teuthology.orchestra.run.smithi078.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T01:32:20.246 INFO:teuthology.orchestra.run.smithi078.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T01:32:20.246 INFO:teuthology.orchestra.run.smithi078.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T01:32:20.247 INFO:teuthology.orchestra.run.smithi078.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T01:32:20.247 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T01:32:20.247 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T01:32:20.247 INFO:teuthology.orchestra.run.smithi078.stdout:Installing weak dependencies: 2024-04-25T01:32:20.247 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T01:32:20.247 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:32:20.247 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-25T01:32:20.247 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================= 2024-04-25T01:32:20.247 INFO:teuthology.orchestra.run.smithi078.stdout:Install 148 Packages 2024-04-25T01:32:20.247 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrade 4 Packages 2024-04-25T01:32:20.247 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:32:20.248 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 220 M 2024-04-25T01:32:20.248 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-25T01:32:20.429 INFO:teuthology.orchestra.run.smithi196.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 330 kB/s | 94 kB 00:00 2024-04-25T01:32:20.455 INFO:teuthology.orchestra.run.smithi196.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 224 kB/s | 64 kB 00:00 2024-04-25T01:32:20.663 INFO:teuthology.orchestra.run.smithi196.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 107 kB/s | 22 kB 00:00 2024-04-25T01:32:20.688 INFO:teuthology.orchestra.run.smithi196.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 173 kB/s | 45 kB 00:00 2024-04-25T01:32:20.955 INFO:teuthology.orchestra.run.smithi196.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-04-25T01:32:21.064 INFO:teuthology.orchestra.run.smithi078.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-04-25T01:32:21.205 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-25T01:32:21.314 INFO:teuthology.orchestra.run.smithi078.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 3.5 MB/s | 890 kB 00:00 2024-04-25T01:32:21.365 INFO:teuthology.orchestra.run.smithi078.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 2.8 MB/s | 145 kB 00:00 2024-04-25T01:32:21.397 INFO:teuthology.orchestra.run.smithi196.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 774 kB/s | 548 kB 00:00 2024-04-25T01:32:21.589 INFO:teuthology.orchestra.run.smithi196.stdout:(105/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.1 MB/s | 16 MB 00:02 2024-04-25T01:32:21.614 INFO:teuthology.orchestra.run.smithi196.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 90 kB/s | 19 kB 00:00 2024-04-25T01:32:21.640 INFO:teuthology.orchestra.run.smithi196.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 451 kB/s | 309 kB 00:00 2024-04-25T01:32:21.723 INFO:teuthology.orchestra.run.smithi196.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-04-25T01:32:21.731 INFO:teuthology.orchestra.run.smithi078.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 6.4 MB/s | 2.4 MB 00:00 2024-04-25T01:32:21.757 INFO:teuthology.orchestra.run.smithi196.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-04-25T01:32:21.798 INFO:teuthology.orchestra.run.smithi078.stdout:(5/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 5.9 MB/s | 5.4 MB 00:00 2024-04-25T01:32:21.832 INFO:teuthology.orchestra.run.smithi196.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-04-25T01:32:21.899 INFO:teuthology.orchestra.run.smithi196.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-25T01:32:22.015 INFO:teuthology.orchestra.run.smithi078.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 5.3 MB/s | 1.5 MB 00:00 2024-04-25T01:32:22.041 INFO:teuthology.orchestra.run.smithi196.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-04-25T01:32:22.141 INFO:teuthology.orchestra.run.smithi196.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-04-25T01:32:22.166 INFO:teuthology.orchestra.run.smithi196.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 347 kB/s | 43 kB 00:00 2024-04-25T01:32:22.250 INFO:teuthology.orchestra.run.smithi196.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-04-25T01:32:22.275 INFO:teuthology.orchestra.run.smithi196.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-25T01:32:22.299 INFO:teuthology.orchestra.run.smithi078.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 9.4 MB/s | 4.7 MB 00:00 2024-04-25T01:32:22.401 INFO:teuthology.orchestra.run.smithi196.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-04-25T01:32:22.426 INFO:teuthology.orchestra.run.smithi196.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-04-25T01:32:22.518 INFO:teuthology.orchestra.run.smithi196.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-25T01:32:22.543 INFO:teuthology.orchestra.run.smithi196.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-04-25T01:32:22.602 INFO:teuthology.orchestra.run.smithi196.stdout:(121/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:01 2024-04-25T01:32:22.635 INFO:teuthology.orchestra.run.smithi196.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-25T01:32:22.728 INFO:teuthology.orchestra.run.smithi196.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-25T01:32:22.753 INFO:teuthology.orchestra.run.smithi196.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-04-25T01:32:22.836 INFO:teuthology.orchestra.run.smithi196.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.0 MB/s | 2.0 MB 00:00 2024-04-25T01:32:22.862 INFO:teuthology.orchestra.run.smithi196.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 132 kB/s | 18 kB 00:00 2024-04-25T01:32:22.887 INFO:teuthology.orchestra.run.smithi196.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 173 kB/s | 23 kB 00:00 2024-04-25T01:32:22.946 INFO:teuthology.orchestra.run.smithi196.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-04-25T01:32:22.971 INFO:teuthology.orchestra.run.smithi196.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-04-25T01:32:23.013 INFO:teuthology.orchestra.run.smithi196.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-04-25T01:32:23.054 INFO:teuthology.orchestra.run.smithi196.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-04-25T01:32:23.080 INFO:teuthology.orchestra.run.smithi196.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 725 kB/s | 79 kB 00:00 2024-04-25T01:32:23.122 INFO:teuthology.orchestra.run.smithi196.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-04-25T01:32:23.172 INFO:teuthology.orchestra.run.smithi196.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-25T01:32:23.197 INFO:teuthology.orchestra.run.smithi196.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-04-25T01:32:23.231 INFO:teuthology.orchestra.run.smithi196.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 826 kB/s | 90 kB 00:00 2024-04-25T01:32:23.281 INFO:teuthology.orchestra.run.smithi196.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-04-25T01:32:23.306 INFO:teuthology.orchestra.run.smithi196.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-04-25T01:32:23.340 INFO:teuthology.orchestra.run.smithi196.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T01:32:23.357 INFO:teuthology.orchestra.run.smithi078.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 8.5 MB/s | 21 MB 00:02 2024-04-25T01:32:23.381 INFO:teuthology.orchestra.run.smithi152.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:32:23.381 INFO:teuthology.orchestra.run.smithi152.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T01:32:23.390 INFO:teuthology.orchestra.run.smithi196.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-04-25T01:32:23.392 INFO:teuthology.orchestra.run.smithi152.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T01:32:23.393 INFO:teuthology.orchestra.run.smithi152.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T01:32:23.394 INFO:teuthology.orchestra.run.smithi152.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T01:32:23.394 INFO:teuthology.orchestra.run.smithi152.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T01:32:23.396 INFO:teuthology.orchestra.run.smithi152.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T01:32:23.398 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T01:32:23.399 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T01:32:23.415 INFO:teuthology.orchestra.run.smithi196.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-04-25T01:32:23.449 INFO:teuthology.orchestra.run.smithi196.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 468 kB/s | 51 kB 00:00 2024-04-25T01:32:23.458 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-04-25T01:32:23.474 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================= 2024-04-25T01:32:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-04-25T01:32:23.475 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================= 2024-04-25T01:32:23.475 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-04-25T01:32:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T01:32:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T01:32:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T01:32:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T01:32:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T01:32:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T01:32:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T01:32:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T01:32:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T01:32:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T01:32:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T01:32:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T01:32:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T01:32:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T01:32:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T01:32:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T01:32:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T01:32:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T01:32:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T01:32:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T01:32:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T01:32:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T01:32:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T01:32:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T01:32:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T01:32:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T01:32:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T01:32:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T01:32:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T01:32:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T01:32:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T01:32:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T01:32:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T01:32:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T01:32:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:32:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T01:32:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T01:32:23.478 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrading: 2024-04-25T01:32:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T01:32:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T01:32:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T01:32:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T01:32:23.478 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-04-25T01:32:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T01:32:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T01:32:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T01:32:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T01:32:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T01:32:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T01:32:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T01:32:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T01:32:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T01:32:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T01:32:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T01:32:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T01:32:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T01:32:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T01:32:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T01:32:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T01:32:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T01:32:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T01:32:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T01:32:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T01:32:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T01:32:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T01:32:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T01:32:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T01:32:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T01:32:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T01:32:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T01:32:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T01:32:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T01:32:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T01:32:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T01:32:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T01:32:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T01:32:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T01:32:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T01:32:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T01:32:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T01:32:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T01:32:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T01:32:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T01:32:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T01:32:23.482 INFO:teuthology.orchestra.run.smithi152.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T01:32:23.482 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T01:32:23.482 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T01:32:23.482 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T01:32:23.482 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T01:32:23.482 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T01:32:23.482 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T01:32:23.482 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T01:32:23.482 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T01:32:23.482 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T01:32:23.482 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T01:32:23.483 INFO:teuthology.orchestra.run.smithi152.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T01:32:23.483 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T01:32:23.483 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T01:32:23.483 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T01:32:23.483 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T01:32:23.483 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T01:32:23.483 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T01:32:23.483 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T01:32:23.483 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T01:32:23.483 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T01:32:23.483 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T01:32:23.483 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T01:32:23.483 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T01:32:23.484 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T01:32:23.484 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T01:32:23.484 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T01:32:23.484 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T01:32:23.484 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T01:32:23.484 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T01:32:23.484 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T01:32:23.484 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T01:32:23.484 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T01:32:23.484 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T01:32:23.484 INFO:teuthology.orchestra.run.smithi152.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T01:32:23.484 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T01:32:23.484 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T01:32:23.484 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T01:32:23.485 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T01:32:23.485 INFO:teuthology.orchestra.run.smithi152.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T01:32:23.485 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T01:32:23.485 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T01:32:23.485 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T01:32:23.485 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T01:32:23.485 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T01:32:23.485 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T01:32:23.485 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T01:32:23.485 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T01:32:23.485 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T01:32:23.485 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T01:32:23.485 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T01:32:23.485 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T01:32:23.486 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T01:32:23.486 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T01:32:23.486 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T01:32:23.486 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T01:32:23.486 INFO:teuthology.orchestra.run.smithi152.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T01:32:23.486 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T01:32:23.486 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T01:32:23.486 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T01:32:23.486 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T01:32:23.486 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T01:32:23.486 INFO:teuthology.orchestra.run.smithi152.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T01:32:23.486 INFO:teuthology.orchestra.run.smithi152.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T01:32:23.486 INFO:teuthology.orchestra.run.smithi152.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T01:32:23.486 INFO:teuthology.orchestra.run.smithi152.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T01:32:23.487 INFO:teuthology.orchestra.run.smithi152.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T01:32:23.487 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T01:32:23.487 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T01:32:23.487 INFO:teuthology.orchestra.run.smithi152.stdout:Installing weak dependencies: 2024-04-25T01:32:23.487 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T01:32:23.487 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:32:23.487 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-04-25T01:32:23.487 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================= 2024-04-25T01:32:23.487 INFO:teuthology.orchestra.run.smithi152.stdout:Install 148 Packages 2024-04-25T01:32:23.487 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrade 4 Packages 2024-04-25T01:32:23.487 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:32:23.488 INFO:teuthology.orchestra.run.smithi078.stdout:(9/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 8.6 MB/s | 10 MB 00:01 2024-04-25T01:32:23.488 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 220 M 2024-04-25T01:32:23.488 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-04-25T01:32:23.499 INFO:teuthology.orchestra.run.smithi196.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-25T01:32:23.508 INFO:teuthology.orchestra.run.smithi078.stdout:(10/152): ceph-selinux-19.0.0-2445.gd2c45d97.el 167 kB/s | 25 kB 00:00 2024-04-25T01:32:23.525 INFO:teuthology.orchestra.run.smithi196.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 822 kB/s | 90 kB 00:00 2024-04-25T01:32:23.567 INFO:teuthology.orchestra.run.smithi196.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-25T01:32:23.575 INFO:teuthology.orchestra.run.smithi078.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 3.3 MB/s | 222 kB 00:00 2024-04-25T01:32:23.608 INFO:teuthology.orchestra.run.smithi196.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-04-25T01:32:23.642 INFO:teuthology.orchestra.run.smithi078.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 407 kB/s | 27 kB 00:00 2024-04-25T01:32:23.642 INFO:teuthology.orchestra.run.smithi196.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-25T01:32:23.709 INFO:teuthology.orchestra.run.smithi078.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 475 kB/s | 32 kB 00:00 2024-04-25T01:32:23.709 INFO:teuthology.orchestra.run.smithi196.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T01:32:23.851 INFO:teuthology.orchestra.run.smithi078.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 5.1 MB/s | 745 kB 00:00 2024-04-25T01:32:23.951 INFO:teuthology.orchestra.run.smithi078.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.6 MB/s | 163 kB 00:00 2024-04-25T01:32:24.043 INFO:teuthology.orchestra.run.smithi078.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.3 MB/s | 127 kB 00:00 2024-04-25T01:32:24.209 INFO:teuthology.orchestra.run.smithi196.stdout:(149/152): librados2-19.0.0-2445.gd2c45d97.el9. 5.6 MB/s | 3.4 MB 00:00 2024-04-25T01:32:24.243 INFO:teuthology.orchestra.run.smithi196.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 2.5 MB/s | 1.3 MB 00:00 2024-04-25T01:32:24.252 INFO:teuthology.orchestra.run.smithi078.stdout:(17/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 7.5 MB/s | 17 MB 00:02 2024-04-25T01:32:24.286 INFO:teuthology.orchestra.run.smithi078.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 2.0 MB/s | 501 kB 00:00 2024-04-25T01:32:24.361 INFO:teuthology.orchestra.run.smithi078.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 601 kB/s | 45 kB 00:00 2024-04-25T01:32:24.453 INFO:teuthology.orchestra.run.smithi078.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.4 MB/s | 133 kB 00:00 2024-04-25T01:32:24.474 INFO:teuthology.orchestra.run.smithi152.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-25T01:32:24.553 INFO:teuthology.orchestra.run.smithi078.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.6 MB/s | 162 kB 00:00 2024-04-25T01:32:24.662 INFO:teuthology.orchestra.run.smithi078.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.9 MB/s | 321 kB 00:00 2024-04-25T01:32:24.734 INFO:teuthology.orchestra.run.smithi196.stdout:(151/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 2.9 MB/s | 3.1 MB 00:01 2024-04-25T01:32:24.746 INFO:teuthology.orchestra.run.smithi078.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.5 MB/s | 299 kB 00:00 2024-04-25T01:32:24.821 INFO:teuthology.orchestra.run.smithi078.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.3 MB/s | 99 kB 00:00 2024-04-25T01:32:24.843 INFO:teuthology.orchestra.run.smithi196.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.8 MB/s | 2.4 MB 00:00 2024-04-25T01:32:24.847 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:32:24.847 INFO:teuthology.orchestra.run.smithi196.stdout:Total 12 MB/s | 220 MB 00:18 2024-04-25T01:32:24.896 INFO:teuthology.orchestra.run.smithi078.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T01:32:24.980 INFO:teuthology.orchestra.run.smithi078.stdout:(26/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 7.3 MB/s | 5.3 MB 00:00 2024-04-25T01:32:25.072 INFO:teuthology.orchestra.run.smithi078.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.8 MB/s | 171 kB 00:00 2024-04-25T01:32:25.155 INFO:teuthology.orchestra.run.smithi078.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 381 kB/s | 32 kB 00:00 2024-04-25T01:32:25.239 INFO:teuthology.orchestra.run.smithi078.stdout:(29/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 9.0 MB/s | 3.1 MB 00:00 2024-04-25T01:32:25.264 INFO:teuthology.orchestra.run.smithi078.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.3 MB/s | 141 kB 00:00 2024-04-25T01:32:25.532 INFO:teuthology.orchestra.run.smithi152.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 841 kB/s | 890 kB 00:01 2024-04-25T01:32:25.723 INFO:teuthology.orchestra.run.smithi078.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 7.3 MB/s | 3.5 MB 00:00 2024-04-25T01:32:25.765 INFO:teuthology.orchestra.run.smithi152.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 621 kB/s | 145 kB 00:00 2024-04-25T01:32:25.823 INFO:teuthology.orchestra.run.smithi078.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.4 MB/s | 249 kB 00:00 2024-04-25T01:32:25.915 INFO:teuthology.orchestra.run.smithi078.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 539 kB/s | 49 kB 00:00 2024-04-25T01:32:26.007 INFO:teuthology.orchestra.run.smithi078.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 167 kB/s | 15 kB 00:00 2024-04-25T01:32:26.080 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-04-25T01:32:26.108 INFO:teuthology.orchestra.run.smithi078.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.6 MB/s | 264 kB 00:00 2024-04-25T01:32:26.226 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-04-25T01:32:26.226 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-04-25T01:32:26.341 INFO:teuthology.orchestra.run.smithi078.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 3.2 MB/s | 762 kB 00:00 2024-04-25T01:32:26.408 INFO:teuthology.orchestra.run.smithi078.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 274 kB/s | 18 kB 00:00 2024-04-25T01:32:26.575 INFO:teuthology.orchestra.run.smithi078.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 482 kB/s | 80 kB 00:00 2024-04-25T01:32:26.700 INFO:teuthology.orchestra.run.smithi078.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-04-25T01:32:26.826 INFO:teuthology.orchestra.run.smithi078.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.1 MB/s | 803 kB 00:00 2024-04-25T01:32:26.892 INFO:teuthology.orchestra.run.smithi078.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-04-25T01:32:26.943 INFO:teuthology.orchestra.run.smithi078.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 807 kB/s | 195 kB 00:00 2024-04-25T01:32:27.001 INFO:teuthology.orchestra.run.smithi078.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-25T01:32:27.027 INFO:teuthology.orchestra.run.smithi078.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-25T01:32:27.060 INFO:teuthology.orchestra.run.smithi078.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T01:32:27.119 INFO:teuthology.orchestra.run.smithi078.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-25T01:32:27.284 INFO:teuthology.orchestra.run.smithi152.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 1.6 MB/s | 2.4 MB 00:01 2024-04-25T01:32:27.366 INFO:teuthology.orchestra.run.smithi152.stdout:(5/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-04-25T01:32:27.620 INFO:teuthology.orchestra.run.smithi078.stdout:(47/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 12 MB/s | 48 MB 00:04 2024-04-25T01:32:27.645 INFO:teuthology.orchestra.run.smithi078.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 367 kB/s | 215 kB 00:00 2024-04-25T01:32:27.841 INFO:teuthology.orchestra.run.smithi152.stdout:(6/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 5.6 MB/s | 21 MB 00:03 2024-04-25T01:32:27.891 INFO:teuthology.orchestra.run.smithi152.stdout:(7/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-04-25T01:32:27.988 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-04-25T01:32:27.988 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-04-25T01:32:28.079 INFO:teuthology.orchestra.run.smithi078.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 247 kB/s | 107 kB 00:00 2024-04-25T01:32:28.354 INFO:teuthology.orchestra.run.smithi078.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 78 kB/s | 21 kB 00:00 2024-04-25T01:32:28.725 INFO:teuthology.orchestra.run.smithi152.stdout:(8/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 19 MB/s | 17 MB 00:00 2024-04-25T01:32:28.792 INFO:teuthology.orchestra.run.smithi152.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 379 kB/s | 25 kB 00:00 2024-04-25T01:32:29.254 INFO:teuthology.orchestra.run.smithi078.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 323 kB/s | 689 kB 00:02 2024-04-25T01:32:29.362 INFO:teuthology.orchestra.run.smithi078.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 311 kB/s | 313 kB 00:01 2024-04-25T01:32:29.479 INFO:teuthology.orchestra.run.smithi078.stdout:(53/152): bison-3.7.4-5.el9.x86_64.rpm 507 kB/s | 943 kB 00:01 2024-04-25T01:32:29.538 INFO:teuthology.orchestra.run.smithi078.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 115 kB/s | 33 kB 00:00 2024-04-25T01:32:29.730 INFO:teuthology.orchestra.run.smithi078.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 58 kB/s | 11 kB 00:00 2024-04-25T01:32:29.772 INFO:teuthology.orchestra.run.smithi078.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 62 kB/s | 18 kB 00:00 2024-04-25T01:32:29.834 INFO:teuthology.orchestra.run.smithi152.stdout:(10/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x8 1.9 MB/s | 4.7 MB 00:02 2024-04-25T01:32:30.067 INFO:teuthology.orchestra.run.smithi152.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 953 kB/s | 222 kB 00:00 2024-04-25T01:32:30.114 INFO:teuthology.orchestra.run.smithi078.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 132 kB/s | 45 kB 00:00 2024-04-25T01:32:30.244 INFO:teuthology.orchestra.run.smithi152.stdout:(12/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 4.4 MB/s | 10 MB 00:02 2024-04-25T01:32:30.269 INFO:teuthology.orchestra.run.smithi152.stdout:(13/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 135 kB/s | 27 kB 00:00 2024-04-25T01:32:30.311 INFO:teuthology.orchestra.run.smithi152.stdout:(14/152): libcephfs-devel-19.0.0-2445.gd2c45d97 476 kB/s | 32 kB 00:00 2024-04-25T01:32:30.339 INFO:teuthology.orchestra.run.smithi078.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 263 kB/s | 160 kB 00:00 2024-04-25T01:32:30.419 INFO:teuthology.orchestra.run.smithi152.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.5 MB/s | 163 kB 00:00 2024-04-25T01:32:30.536 INFO:teuthology.orchestra.run.smithi152.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.1 MB/s | 127 kB 00:00 2024-04-25T01:32:30.637 INFO:teuthology.orchestra.run.smithi152.stdout:(17/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 2.0 MB/s | 745 kB 00:00 2024-04-25T01:32:30.745 INFO:teuthology.orchestra.run.smithi152.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 2.3 MB/s | 501 kB 00:00 2024-04-25T01:32:30.854 INFO:teuthology.orchestra.run.smithi152.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 417 kB/s | 45 kB 00:00 2024-04-25T01:32:31.296 INFO:teuthology.orchestra.run.smithi152.stdout:(20/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 19 MB/s | 48 MB 00:02 2024-04-25T01:32:31.321 INFO:teuthology.orchestra.run.smithi152.stdout:(21/152): python3-ceph-common-19.0.0-2445.gd2c4 285 kB/s | 133 kB 00:00 2024-04-25T01:32:31.355 INFO:teuthology.orchestra.run.smithi152.stdout:(22/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.7 MB/s | 162 kB 00:00 2024-04-25T01:32:31.389 INFO:teuthology.orchestra.run.smithi078.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 555 kB/s | 582 kB 00:01 2024-04-25T01:32:31.405 INFO:teuthology.orchestra.run.smithi152.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 5.9 MB/s | 299 kB 00:00 2024-04-25T01:32:31.430 INFO:teuthology.orchestra.run.smithi152.stdout:(24/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.9 MB/s | 321 kB 00:00 2024-04-25T01:32:31.455 INFO:teuthology.orchestra.run.smithi152.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.9 MB/s | 99 kB 00:00 2024-04-25T01:32:31.497 INFO:teuthology.orchestra.run.smithi152.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-25T01:32:31.548 INFO:teuthology.orchestra.run.smithi078.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 462 kB/s | 662 kB 00:01 2024-04-25T01:32:31.614 INFO:teuthology.orchestra.run.smithi152.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.4 MB/s | 171 kB 00:00 2024-04-25T01:32:31.656 INFO:teuthology.orchestra.run.smithi078.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 95 kB/s | 25 kB 00:00 2024-04-25T01:32:31.673 INFO:teuthology.orchestra.run.smithi152.stdout:(28/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 14 MB/s | 3.1 MB 00:00 2024-04-25T01:32:31.698 INFO:teuthology.orchestra.run.smithi152.stdout:(29/152): ceph-grafana-dashboards-19.0.0-2445.g 379 kB/s | 32 kB 00:00 2024-04-25T01:32:31.724 INFO:teuthology.orchestra.run.smithi152.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.7 MB/s | 141 kB 00:00 2024-04-25T01:32:31.882 INFO:teuthology.orchestra.run.smithi078.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 142 kB/s | 47 kB 00:00 2024-04-25T01:32:32.191 INFO:teuthology.orchestra.run.smithi152.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 16 MB/s | 7.4 MB 00:00 2024-04-25T01:32:32.258 INFO:teuthology.orchestra.run.smithi152.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 3.7 MB/s | 249 kB 00:00 2024-04-25T01:32:32.316 INFO:teuthology.orchestra.run.smithi152.stdout:(33/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 5.7 MB/s | 3.5 MB 00:00 2024-04-25T01:32:32.324 INFO:teuthology.orchestra.run.smithi078.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 438 kB/s | 292 kB 00:00 2024-04-25T01:32:32.342 INFO:teuthology.orchestra.run.smithi152.stdout:(34/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 590 kB/s | 49 kB 00:00 2024-04-25T01:32:32.392 INFO:teuthology.orchestra.run.smithi152.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 5.2 MB/s | 264 kB 00:00 2024-04-25T01:32:32.417 INFO:teuthology.orchestra.run.smithi152.stdout:(36/152): ceph-prometheus-alerts-19.0.0-2445.gd 152 kB/s | 15 kB 00:00 2024-04-25T01:32:32.459 INFO:teuthology.orchestra.run.smithi152.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 435 kB/s | 18 kB 00:00 2024-04-25T01:32:32.540 INFO:teuthology.orchestra.run.smithi078.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 286 kB/s | 188 kB 00:00 2024-04-25T01:32:32.560 INFO:teuthology.orchestra.run.smithi152.stdout:(38/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 4.4 MB/s | 762 kB 00:00 2024-04-25T01:32:32.582 INFO:teuthology.orchestra.run.smithi078.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 136 kB/s | 35 kB 00:00 2024-04-25T01:32:32.660 INFO:teuthology.orchestra.run.smithi152.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 401 kB/s | 80 kB 00:00 2024-04-25T01:32:32.727 INFO:teuthology.orchestra.run.smithi152.stdout:(40/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 2.5 MB/s | 5.3 MB 00:02 2024-04-25T01:32:32.952 INFO:teuthology.orchestra.run.smithi152.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-04-25T01:32:32.978 INFO:teuthology.orchestra.run.smithi152.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 616 kB/s | 195 kB 00:00 2024-04-25T01:32:32.999 INFO:teuthology.orchestra.run.smithi078.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 842 kB/s | 3.0 MB 00:03 2024-04-25T01:32:33.053 INFO:teuthology.orchestra.run.smithi152.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 778 kB/s | 253 kB 00:00 2024-04-25T01:32:33.261 INFO:teuthology.orchestra.run.smithi152.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 375 kB/s | 106 kB 00:00 2024-04-25T01:32:33.303 INFO:teuthology.orchestra.run.smithi152.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 540 kB/s | 135 kB 00:00 2024-04-25T01:32:33.354 INFO:teuthology.orchestra.run.smithi152.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-04-25T01:32:33.496 INFO:teuthology.orchestra.run.smithi152.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 534 kB/s | 125 kB 00:00 2024-04-25T01:32:33.579 INFO:teuthology.orchestra.run.smithi152.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 779 kB/s | 215 kB 00:00 2024-04-25T01:32:34.071 INFO:teuthology.orchestra.run.smithi152.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 217 kB/s | 107 kB 00:00 2024-04-25T01:32:34.090 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-04-25T01:32:34.163 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T01:32:34.201 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T01:32:34.213 INFO:teuthology.orchestra.run.smithi152.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 152 kB/s | 21 kB 00:00 2024-04-25T01:32:34.238 INFO:teuthology.orchestra.run.smithi152.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 779 kB/s | 689 kB 00:00 2024-04-25T01:32:34.339 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T01:32:34.347 INFO:teuthology.orchestra.run.smithi152.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-04-25T01:32:34.425 INFO:teuthology.orchestra.run.smithi078.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 1.1 MB/s | 1.6 MB 00:01 2024-04-25T01:32:34.481 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T01:32:34.491 INFO:teuthology.orchestra.run.smithi078.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 2.4 MB/s | 4.7 MB 00:01 2024-04-25T01:32:34.497 INFO:teuthology.orchestra.run.smithi152.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-04-25T01:32:34.506 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T01:32:34.522 INFO:teuthology.orchestra.run.smithi152.stdout:(54/152): bison-3.7.4-5.el9.x86_64.rpm 918 kB/s | 943 kB 00:01 2024-04-25T01:32:34.550 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T01:32:34.623 INFO:teuthology.orchestra.run.smithi152.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 145 kB/s | 18 kB 00:00 2024-04-25T01:32:34.657 INFO:teuthology.orchestra.run.smithi152.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 83 kB/s | 11 kB 00:00 2024-04-25T01:32:34.807 INFO:teuthology.orchestra.run.smithi152.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 302 kB/s | 45 kB 00:00 2024-04-25T01:32:34.880 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T01:32:34.990 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:32:34.991 INFO:teuthology.orchestra.run.smithi078.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 266 kB/s | 133 kB 00:00 2024-04-25T01:32:35.040 INFO:teuthology.orchestra.run.smithi152.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-04-25T01:32:35.110 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:32:35.133 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:32:35.194 INFO:teuthology.orchestra.run.smithi152.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 3.7 MB/s | 582 kB 00:00 2024-04-25T01:32:35.207 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:32:35.237 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:32:35.249 INFO:teuthology.orchestra.run.smithi152.stdout:(60/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 256 kB/s | 160 kB 00:00 2024-04-25T01:32:35.308 INFO:teuthology.orchestra.run.smithi152.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 223 kB/s | 25 kB 00:00 2024-04-25T01:32:35.358 INFO:teuthology.orchestra.run.smithi152.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-04-25T01:32:35.392 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T01:32:35.425 INFO:teuthology.orchestra.run.smithi152.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-04-25T01:32:35.442 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T01:32:35.470 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:32:35.534 INFO:teuthology.orchestra.run.smithi152.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 324 kB/s | 35 kB 00:00 2024-04-25T01:32:35.559 INFO:teuthology.orchestra.run.smithi152.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 938 kB/s | 188 kB 00:00 2024-04-25T01:32:35.572 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:32:35.596 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:32:35.633 INFO:teuthology.orchestra.run.smithi078.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 754 kB/s | 911 kB 00:01 2024-04-25T01:32:35.658 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:32:35.680 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:32:35.700 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:32:35.897 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:32:35.930 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:32:36.035 INFO:teuthology.orchestra.run.smithi152.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 9.5 MB/s | 4.7 MB 00:00 2024-04-25T01:32:36.036 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T01:32:36.132 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T01:32:36.171 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T01:32:36.205 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T01:32:36.269 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T01:32:36.312 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T01:32:36.325 INFO:teuthology.orchestra.run.smithi078.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.2 MB/s | 4.4 MB 00:03 2024-04-25T01:32:36.356 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T01:32:36.410 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T01:32:36.441 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T01:32:36.510 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T01:32:36.563 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T01:32:36.568 INFO:teuthology.orchestra.run.smithi152.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 3.0 MB/s | 1.6 MB 00:00 2024-04-25T01:32:36.593 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T01:32:36.629 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T01:32:36.705 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T01:32:36.739 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T01:32:36.742 INFO:teuthology.orchestra.run.smithi078.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 210 kB/s | 87 kB 00:00 2024-04-25T01:32:36.783 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T01:32:36.812 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T01:32:36.880 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T01:32:36.935 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T01:32:37.008 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T01:32:37.141 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T01:32:37.203 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T01:32:37.484 INFO:teuthology.orchestra.run.smithi078.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 558 kB/s | 1.0 MB 00:01 2024-04-25T01:32:37.485 INFO:teuthology.orchestra.run.smithi152.stdout:(68/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 976 kB/s | 3.0 MB 00:03 2024-04-25T01:32:37.594 INFO:teuthology.orchestra.run.smithi152.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-25T01:32:37.724 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T01:32:37.806 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T01:32:37.844 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T01:32:37.887 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T01:32:37.925 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T01:32:37.967 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T01:32:38.128 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T01:32:38.178 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T01:32:38.233 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T01:32:38.284 INFO:teuthology.orchestra.run.smithi078.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 305 kB/s | 244 kB 00:00 2024-04-25T01:32:38.305 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T01:32:38.337 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T01:32:38.373 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T01:32:38.419 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T01:32:38.456 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T01:32:38.498 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T01:32:38.541 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T01:32:38.676 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T01:32:38.677 INFO:teuthology.orchestra.run.smithi152.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.4 MB/s | 4.4 MB 00:03 2024-04-25T01:32:38.730 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T01:32:38.760 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T01:32:38.801 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T01:32:38.896 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T01:32:38.934 INFO:teuthology.orchestra.run.smithi078.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 381 kB/s | 248 kB 00:00 2024-04-25T01:32:38.935 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T01:32:38.974 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T01:32:39.000 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T01:32:39.020 INFO:teuthology.orchestra.run.smithi078.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 1.4 MB/s | 5.8 MB 00:04 2024-04-25T01:32:39.033 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T01:32:39.106 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T01:32:39.234 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T01:32:39.237 INFO:teuthology.orchestra.run.smithi078.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 2.4 MB/s | 6.0 MB 00:02 2024-04-25T01:32:39.261 INFO:teuthology.orchestra.run.smithi152.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 1.7 MB/s | 1.0 MB 00:00 2024-04-25T01:32:39.262 INFO:teuthology.orchestra.run.smithi078.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 78 kB/s | 26 kB 00:00 2024-04-25T01:32:39.265 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T01:32:39.288 INFO:teuthology.orchestra.run.smithi078.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 72 kB/s | 19 kB 00:00 2024-04-25T01:32:39.293 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T01:32:39.327 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T01:32:39.361 INFO:teuthology.orchestra.run.smithi152.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 876 kB/s | 87 kB 00:00 2024-04-25T01:32:39.366 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T01:32:39.398 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T01:32:39.427 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T01:32:39.462 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T01:32:39.495 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T01:32:39.529 INFO:teuthology.orchestra.run.smithi078.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 130 kB/s | 35 kB 00:00 2024-04-25T01:32:39.545 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T01:32:39.580 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T01:32:39.666 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T01:32:39.710 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T01:32:39.750 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T01:32:39.806 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T01:32:39.888 INFO:teuthology.orchestra.run.smithi078.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 265 kB/s | 172 kB 00:00 2024-04-25T01:32:39.911 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T01:32:39.959 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T01:32:40.122 INFO:teuthology.orchestra.run.smithi078.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 423 kB/s | 250 kB 00:00 2024-04-25T01:32:40.261 INFO:teuthology.orchestra.run.smithi152.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 2.2 MB/s | 5.8 MB 00:02 2024-04-25T01:32:40.370 INFO:teuthology.orchestra.run.smithi152.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-04-25T01:32:40.478 INFO:teuthology.orchestra.run.smithi152.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-25T01:32:40.480 INFO:teuthology.orchestra.run.smithi078.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 215 kB/s | 77 kB 00:00 2024-04-25T01:32:40.578 INFO:teuthology.orchestra.run.smithi152.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-04-25T01:32:40.614 INFO:teuthology.orchestra.run.smithi078.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 306 kB/s | 222 kB 00:00 2024-04-25T01:32:40.679 INFO:teuthology.orchestra.run.smithi152.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-04-25T01:32:40.682 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T01:32:40.723 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T01:32:40.756 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T01:32:40.779 INFO:teuthology.orchestra.run.smithi152.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-04-25T01:32:40.855 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T01:32:40.856 INFO:teuthology.orchestra.run.smithi078.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 113 kB/s | 42 kB 00:00 2024-04-25T01:32:40.880 INFO:teuthology.orchestra.run.smithi152.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 347 kB/s | 35 kB 00:00 2024-04-25T01:32:40.923 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T01:32:41.009 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T01:32:41.042 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T01:32:41.218 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T01:32:41.246 INFO:teuthology.orchestra.run.smithi152.stdout:(80/152): postgresql-contrib-13.14-1.el9.x86_64 195 kB/s | 911 kB 00:04 2024-04-25T01:32:41.256 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T01:32:41.333 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T01:32:41.355 INFO:teuthology.orchestra.run.smithi152.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-25T01:32:41.414 INFO:teuthology.orchestra.run.smithi078.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 336 kB/s | 269 kB 00:00 2024-04-25T01:32:41.440 INFO:teuthology.orchestra.run.smithi078.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 272 kB/s | 159 kB 00:00 2024-04-25T01:32:41.472 INFO:teuthology.orchestra.run.smithi152.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-04-25T01:32:41.581 INFO:teuthology.orchestra.run.smithi152.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 714 kB/s | 77 kB 00:00 2024-04-25T01:32:41.689 INFO:teuthology.orchestra.run.smithi152.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-04-25T01:32:41.782 INFO:teuthology.orchestra.run.smithi078.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 149 kB/s | 51 kB 00:00 2024-04-25T01:32:41.806 INFO:teuthology.orchestra.run.smithi152.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-04-25T01:32:41.890 INFO:teuthology.orchestra.run.smithi152.stdout:(86/152): python3-babel-2.9.1-2.el9.noarch.rpm 2.4 MB/s | 6.0 MB 00:02 2024-04-25T01:32:41.915 INFO:teuthology.orchestra.run.smithi152.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-25T01:32:41.979 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T01:32:41.999 INFO:teuthology.orchestra.run.smithi152.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-25T01:32:42.015 INFO:teuthology.orchestra.run.smithi078.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 464 kB/s | 279 kB 00:00 2024-04-25T01:32:42.016 INFO:teuthology.orchestra.run.smithi152.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-04-25T01:32:42.087 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T01:32:42.092 INFO:teuthology.orchestra.run.smithi152.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 580 kB/s | 54 kB 00:00 2024-04-25T01:32:42.124 INFO:teuthology.orchestra.run.smithi078.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 157 kB/s | 54 kB 00:00 2024-04-25T01:32:42.200 INFO:teuthology.orchestra.run.smithi152.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-04-25T01:32:42.284 INFO:teuthology.orchestra.run.smithi152.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.6 MB/s | 5.1 MB 00:01 2024-04-25T01:32:42.309 INFO:teuthology.orchestra.run.smithi152.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 406 kB/s | 44 kB 00:00 2024-04-25T01:32:42.382 INFO:teuthology.orchestra.run.smithi078.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 161 kB/s | 42 kB 00:00 2024-04-25T01:32:42.392 INFO:teuthology.orchestra.run.smithi152.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-04-25T01:32:42.418 INFO:teuthology.orchestra.run.smithi152.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 330 kB/s | 36 kB 00:00 2024-04-25T01:32:42.493 INFO:teuthology.orchestra.run.smithi152.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-04-25T01:32:42.527 INFO:teuthology.orchestra.run.smithi152.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-04-25T01:32:42.594 INFO:teuthology.orchestra.run.smithi152.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 575 kB/s | 58 kB 00:00 2024-04-25T01:32:42.636 INFO:teuthology.orchestra.run.smithi152.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 862 kB/s | 94 kB 00:00 2024-04-25T01:32:42.694 INFO:teuthology.orchestra.run.smithi152.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 636 kB/s | 64 kB 00:00 2024-04-25T01:32:42.724 INFO:teuthology.orchestra.run.smithi078.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 129 kB/s | 44 kB 00:00 2024-04-25T01:32:42.736 INFO:teuthology.orchestra.run.smithi152.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 446 kB/s | 45 kB 00:00 2024-04-25T01:32:42.821 INFO:teuthology.orchestra.run.smithi152.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-04-25T01:32:43.170 INFO:teuthology.orchestra.run.smithi152.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-04-25T01:32:43.266 INFO:teuthology.orchestra.run.smithi078.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.3 MB/s | 5.1 MB 00:03 2024-04-25T01:32:43.311 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T01:32:43.392 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T01:32:43.420 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T01:32:43.442 INFO:teuthology.orchestra.run.smithi078.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 308 kB/s | 221 kB 00:00 2024-04-25T01:32:43.446 INFO:teuthology.orchestra.run.smithi152.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 877 kB/s | 548 kB 00:00 2024-04-25T01:32:43.453 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T01:32:43.525 INFO:teuthology.orchestra.run.smithi078.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 139 kB/s | 36 kB 00:00 2024-04-25T01:32:43.754 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T01:32:43.784 INFO:teuthology.orchestra.run.smithi078.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 139 kB/s | 47 kB 00:00 2024-04-25T01:32:43.791 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:32:43.882 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:32:43.912 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T01:32:43.948 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T01:32:44.109 INFO:teuthology.orchestra.run.smithi078.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 522 kB/s | 305 kB 00:00 2024-04-25T01:32:44.134 INFO:teuthology.orchestra.run.smithi078.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 165 kB/s | 58 kB 00:00 2024-04-25T01:32:44.180 INFO:teuthology.orchestra.run.smithi152.stdout:(105/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.5 MB/s | 16 MB 00:02 2024-04-25T01:32:44.429 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T01:32:44.460 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:32:44.476 INFO:teuthology.orchestra.run.smithi078.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 186 kB/s | 64 kB 00:00 2024-04-25T01:32:44.515 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:32:44.526 INFO:teuthology.orchestra.run.smithi078.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 225 kB/s | 94 kB 00:00 2024-04-25T01:32:44.545 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T01:32:44.596 INFO:teuthology.orchestra.run.smithi152.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 17 kB/s | 19 kB 00:01 2024-04-25T01:32:44.602 INFO:teuthology.orchestra.run.smithi078.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 298 kB/s | 22 kB 00:00 2024-04-25T01:32:44.622 INFO:teuthology.orchestra.run.smithi152.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 213 kB/s | 309 kB 00:01 2024-04-25T01:32:44.735 INFO:teuthology.orchestra.run.smithi078.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-04-25T01:32:44.755 INFO:teuthology.orchestra.run.smithi152.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 364 kB/s | 49 kB 00:00 2024-04-25T01:32:44.819 INFO:teuthology.orchestra.run.smithi078.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 131 kB/s | 45 kB 00:00 2024-04-25T01:32:44.939 INFO:teuthology.orchestra.run.smithi152.stdout:(109/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 76 kB/s | 26 kB 00:00 2024-04-25T01:32:44.964 INFO:teuthology.orchestra.run.smithi152.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 323 kB/s | 67 kB 00:00 2024-04-25T01:32:45.064 INFO:teuthology.orchestra.run.smithi152.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-25T01:32:45.448 INFO:teuthology.orchestra.run.smithi152.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 838 kB 00:00 2024-04-25T01:32:45.503 INFO:teuthology.orchestra.run.smithi078.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 452 kB/s | 309 kB 00:00 2024-04-25T01:32:45.515 INFO:teuthology.orchestra.run.smithi152.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 1.2 MB/s | 548 kB 00:00 2024-04-25T01:32:45.636 INFO:teuthology.orchestra.run.smithi078.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-04-25T01:32:45.662 INFO:teuthology.orchestra.run.smithi078.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 592 kB/s | 548 kB 00:00 2024-04-25T01:32:45.741 INFO:teuthology.orchestra.run.smithi152.stdout:(114/152): libarrow-9.0.0-10.el9.x86_64.rpm 2.8 MB/s | 4.4 MB 00:01 2024-04-25T01:32:45.757 INFO:teuthology.orchestra.run.smithi152.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 133 kB/s | 32 kB 00:00 2024-04-25T01:32:45.783 INFO:teuthology.orchestra.run.smithi152.stdout:(116/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 130 kB/s | 43 kB 00:00 2024-04-25T01:32:45.787 INFO:teuthology.orchestra.run.smithi078.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 207 kB/s | 26 kB 00:00 2024-04-25T01:32:45.892 INFO:teuthology.orchestra.run.smithi152.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 99 kB/s | 15 kB 00:00 2024-04-25T01:32:45.917 INFO:teuthology.orchestra.run.smithi152.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-04-25T01:32:45.942 INFO:teuthology.orchestra.run.smithi152.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2024-04-25T01:32:46.062 INFO:teuthology.orchestra.run.smithi078.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 176 kB/s | 49 kB 00:00 2024-04-25T01:32:46.171 INFO:teuthology.orchestra.run.smithi078.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-04-25T01:32:46.201 INFO:teuthology.orchestra.run.smithi152.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 981 kB/s | 303 kB 00:00 2024-04-25T01:32:46.226 INFO:teuthology.orchestra.run.smithi152.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 114 kB/s | 35 kB 00:00 2024-04-25T01:32:46.338 INFO:teuthology.orchestra.run.smithi078.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 908 kB/s | 151 kB 00:00 2024-04-25T01:32:46.593 INFO:teuthology.orchestra.run.smithi152.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 316 kB/s | 206 kB 00:00 2024-04-25T01:32:46.618 INFO:teuthology.orchestra.run.smithi152.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 369 kB/s | 144 kB 00:00 2024-04-25T01:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.5 MB/s | 2.0 MB 00:00 2024-04-25T01:32:46.838 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:32:46.875 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:32:46.886 INFO:teuthology.orchestra.run.smithi152.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 37 kB/s | 11 kB 00:00 2024-04-25T01:32:46.911 INFO:teuthology.orchestra.run.smithi152.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 60 kB/s | 18 kB 00:00 2024-04-25T01:32:46.936 INFO:teuthology.orchestra.run.smithi152.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 154 kB/s | 23 kB 00:00 2024-04-25T01:32:46.978 INFO:teuthology.orchestra.run.smithi152.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-25T01:32:47.012 INFO:teuthology.orchestra.run.smithi152.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 195 kB/s | 20 kB 00:00 2024-04-25T01:32:47.120 INFO:teuthology.orchestra.run.smithi152.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 326 kB/s | 46 kB 00:00 2024-04-25T01:32:47.146 INFO:teuthology.orchestra.run.smithi152.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 590 kB/s | 79 kB 00:00 2024-04-25T01:32:47.179 INFO:teuthology.orchestra.run.smithi152.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-04-25T01:32:47.180 INFO:teuthology.orchestra.run.smithi078.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 995 kB/s | 838 kB 00:00 2024-04-25T01:32:47.271 INFO:teuthology.orchestra.run.smithi152.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 383 kB/s | 58 kB 00:00 2024-04-25T01:32:47.305 INFO:teuthology.orchestra.run.smithi152.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-04-25T01:32:47.330 INFO:teuthology.orchestra.run.smithi152.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 109 kB/s | 16 kB 00:00 2024-04-25T01:32:47.412 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:32:47.461 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:32:47.559 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:32:47.560 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T01:32:47.560 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T01:32:47.560 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:32:47.596 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:32:47.681 INFO:teuthology.orchestra.run.smithi078.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 1.1 MB/s | 548 kB 00:00 2024-04-25T01:32:47.705 INFO:teuthology.orchestra.run.smithi152.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 70 kB/s | 28 kB 00:00 2024-04-25T01:32:47.730 INFO:teuthology.orchestra.run.smithi152.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 77 kB/s | 31 kB 00:00 2024-04-25T01:32:47.748 INFO:teuthology.orchestra.run.smithi152.stdout:(138/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 189 kB/s | 90 kB 00:00 2024-04-25T01:32:47.915 INFO:teuthology.orchestra.run.smithi152.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 322 kB/s | 59 kB 00:00 2024-04-25T01:32:47.931 INFO:teuthology.orchestra.run.smithi078.stdout:(113/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.7 MB/s | 16 MB 00:05 2024-04-25T01:32:47.940 INFO:teuthology.orchestra.run.smithi152.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 186 kB/s | 36 kB 00:00 2024-04-25T01:32:47.956 INFO:teuthology.orchestra.run.smithi078.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 158 kB/s | 43 kB 00:00 2024-04-25T01:32:47.965 INFO:teuthology.orchestra.run.smithi152.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 726 kB/s | 188 kB 00:00 2024-04-25T01:32:48.056 INFO:teuthology.orchestra.run.smithi078.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-25T01:32:48.074 INFO:teuthology.orchestra.run.smithi152.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 321 kB/s | 51 kB 00:00 2024-04-25T01:32:48.099 INFO:teuthology.orchestra.run.smithi152.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-04-25T01:32:48.190 INFO:teuthology.orchestra.run.smithi078.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 124 kB/s | 32 kB 00:00 2024-04-25T01:32:48.282 INFO:teuthology.orchestra.run.smithi078.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 762 kB/s | 172 kB 00:00 2024-04-25T01:32:48.350 INFO:teuthology.orchestra.run.smithi152.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 80 kB/s | 20 kB 00:00 2024-04-25T01:32:48.492 INFO:teuthology.orchestra.run.smithi152.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.0 MB/s | 427 kB 00:00 2024-04-25T01:32:48.667 INFO:teuthology.orchestra.run.smithi152.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 605 kB/s | 192 kB 00:00 2024-04-25T01:32:48.709 INFO:teuthology.orchestra.run.smithi152.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-25T01:32:48.784 INFO:teuthology.orchestra.run.smithi152.stdout:(148/152): python3-websocket-client-1.2.3-2.el9 110 kB/s | 90 kB 00:00 2024-04-25T01:32:48.807 INFO:teuthology.orchestra.run.smithi078.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 577 kB/s | 303 kB 00:00 2024-04-25T01:32:48.941 INFO:teuthology.orchestra.run.smithi078.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 478 kB/s | 358 kB 00:00 2024-04-25T01:32:48.966 INFO:teuthology.orchestra.run.smithi078.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 222 kB/s | 35 kB 00:00 2024-04-25T01:32:49.325 INFO:teuthology.orchestra.run.smithi078.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 536 kB/s | 206 kB 00:00 2024-04-25T01:32:49.376 INFO:teuthology.orchestra.run.smithi152.stdout:(149/152): librados2-19.0.0-2445.gd2c45d97.el9. 4.8 MB/s | 3.4 MB 00:00 2024-04-25T01:32:49.525 INFO:teuthology.orchestra.run.smithi078.stdout:(122/152): libarrow-9.0.0-10.el9.x86_64.rpm 1.1 MB/s | 4.4 MB 00:03 2024-04-25T01:32:49.651 INFO:teuthology.orchestra.run.smithi078.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 444 kB/s | 144 kB 00:00 2024-04-25T01:32:49.667 INFO:teuthology.orchestra.run.smithi078.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-04-25T01:32:49.768 INFO:teuthology.orchestra.run.smithi078.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-25T01:32:49.793 INFO:teuthology.orchestra.run.smithi078.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2024-04-25T01:32:49.876 INFO:teuthology.orchestra.run.smithi078.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-04-25T01:32:49.901 INFO:teuthology.orchestra.run.smithi078.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-04-25T01:32:49.960 INFO:teuthology.orchestra.run.smithi152.stdout:(150/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-04-25T01:32:50.010 INFO:teuthology.orchestra.run.smithi078.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2024-04-25T01:32:50.077 INFO:teuthology.orchestra.run.smithi152.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.0 MB/s | 1.3 MB 00:01 2024-04-25T01:32:50.186 INFO:teuthology.orchestra.run.smithi078.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 449 kB/s | 79 kB 00:00 2024-04-25T01:32:50.286 INFO:teuthology.orchestra.run.smithi078.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 577 kB/s | 58 kB 00:00 2024-04-25T01:32:50.435 INFO:teuthology.orchestra.run.smithi152.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-04-25T01:32:50.439 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:32:50.439 INFO:teuthology.orchestra.run.smithi152.stdout:Total 8.2 MB/s | 220 MB 00:26 2024-04-25T01:32:50.645 INFO:teuthology.orchestra.run.smithi078.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.3 MB/s | 1.0 MB 00:00 2024-04-25T01:32:50.670 INFO:teuthology.orchestra.run.smithi078.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 710 kB/s | 272 kB 00:00 2024-04-25T01:32:50.745 INFO:teuthology.orchestra.run.smithi078.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-25T01:32:50.837 INFO:teuthology.orchestra.run.smithi078.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 539 kB/s | 90 kB 00:00 2024-04-25T01:32:50.862 INFO:teuthology.orchestra.run.smithi078.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-04-25T01:32:50.929 INFO:teuthology.orchestra.run.smithi078.stdout:(137/152): python3-grpcio-1.46.7-10.el9.x86_64. 1.0 MB/s | 2.0 MB 00:01 2024-04-25T01:32:50.955 INFO:teuthology.orchestra.run.smithi078.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 262 kB/s | 31 kB 00:00 2024-04-25T01:32:51.055 INFO:teuthology.orchestra.run.smithi078.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 471 kB/s | 59 kB 00:00 2024-04-25T01:32:51.080 INFO:teuthology.orchestra.run.smithi078.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 286 kB/s | 36 kB 00:00 2024-04-25T01:32:51.097 INFO:teuthology.orchestra.run.smithi078.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 804 kB/s | 188 kB 00:00 2024-04-25T01:32:51.181 INFO:teuthology.orchestra.run.smithi078.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 406 kB/s | 51 kB 00:00 2024-04-25T01:32:51.206 INFO:teuthology.orchestra.run.smithi078.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 825 kB/s | 90 kB 00:00 2024-04-25T01:32:51.323 INFO:teuthology.orchestra.run.smithi078.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-04-25T01:32:51.568 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-04-25T01:32:51.590 INFO:teuthology.orchestra.run.smithi078.stdout:(145/152): python3-webob-1.8.7-6.el9.noarch.rpm 451 kB/s | 230 kB 00:00 2024-04-25T01:32:51.615 INFO:teuthology.orchestra.run.smithi078.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 656 kB/s | 192 kB 00:00 2024-04-25T01:32:51.691 INFO:teuthology.orchestra.run.smithi078.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 838 kB/s | 427 kB 00:00 2024-04-25T01:32:51.712 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-04-25T01:32:51.712 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-04-25T01:32:51.983 INFO:teuthology.orchestra.run.smithi078.stdout:(148/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 11 MB/s | 3.1 MB 00:00 2024-04-25T01:32:52.033 INFO:teuthology.orchestra.run.smithi078.stdout:(149/152): librados2-19.0.0-2445.gd2c45d97.el9. 8.1 MB/s | 3.4 MB 00:00 2024-04-25T01:32:52.250 INFO:teuthology.orchestra.run.smithi078.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 5.1 MB/s | 1.3 MB 00:00 2024-04-25T01:32:52.359 INFO:teuthology.orchestra.run.smithi078.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 2.4 MB 00:00 2024-04-25T01:32:53.317 INFO:teuthology.orchestra.run.smithi078.stdout:(152/152): thrift-0.15.0-2.el9.x86_64.rpm 938 kB/s | 1.6 MB 00:01 2024-04-25T01:32:53.319 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:32:53.320 INFO:teuthology.orchestra.run.smithi078.stdout:Total 6.7 MB/s | 220 MB 00:33 2024-04-25T01:32:53.456 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-04-25T01:32:53.457 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-04-25T01:32:54.429 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-25T01:32:54.572 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-25T01:32:54.573 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-25T01:32:56.330 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-25T01:32:56.331 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-25T01:32:59.085 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:32:59.085 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-04-25T01:32:59.085 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-04-25T01:32:59.086 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-04-25T01:32:59.086 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-04-25T01:32:59.086 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-04-25T01:32:59.086 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-04-25T01:32:59.086 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-04-25T01:32:59.086 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:32:59.264 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:32:59.280 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:32:59.280 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:32:59.280 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T01:32:59.280 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:32:59.624 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-04-25T01:32:59.705 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T01:32:59.744 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T01:32:59.797 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T01:32:59.831 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T01:32:59.857 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T01:32:59.899 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T01:33:00.233 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T01:33:00.341 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:33:00.447 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:33:00.477 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:33:00.550 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:33:00.580 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:33:00.731 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T01:33:00.742 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:33:00.768 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:33:00.768 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:00.768 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T01:33:00.768 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:33:00.768 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:33:00.769 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:33:00.786 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T01:33:00.840 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:33:00.942 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:33:00.965 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:33:00.993 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:33:01.016 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:33:01.016 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:01.016 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T01:33:01.016 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:33:01.016 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:33:01.016 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:33:01.028 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:33:01.050 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:33:01.071 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:33:01.224 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:33:01.258 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:33:01.366 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T01:33:01.452 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:33:01.460 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T01:33:01.474 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:33:01.475 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:01.475 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T01:33:01.475 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:33:01.475 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:33:01.475 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:33:01.490 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T01:33:01.524 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T01:33:01.589 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T01:33:01.599 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T01:33:01.624 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T01:33:01.656 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T01:33:01.668 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T01:33:01.700 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T01:33:01.711 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T01:33:01.728 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T01:33:01.797 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T01:33:01.859 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T01:33:01.889 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T01:33:01.896 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T01:33:01.924 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T01:33:01.944 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T01:33:01.999 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T01:33:02.014 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T01:33:02.035 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T01:33:02.078 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T01:33:02.103 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:33:02.107 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T01:33:02.177 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T01:33:02.230 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T01:33:02.234 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:33:02.269 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T01:33:02.301 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T01:33:02.305 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T01:33:02.306 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-25T01:33:02.329 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T01:33:02.396 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T01:33:02.434 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T01:33:02.437 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T01:33:02.487 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T01:33:02.491 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T01:33:02.530 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T01:33:02.555 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T01:33:02.598 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T01:33:02.743 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:33:02.760 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:33:02.792 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T01:33:02.845 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T01:33:02.875 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:33:02.930 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T01:33:03.007 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T01:33:03.065 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:33:03.095 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T01:33:03.125 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T01:33:03.168 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T01:33:03.186 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:33:03.205 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T01:33:03.224 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:33:03.247 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T01:33:03.316 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:33:03.345 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:33:03.421 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T01:33:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T01:33:03.493 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T01:33:03.538 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T01:33:03.551 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T01:33:03.579 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:33:03.602 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T01:33:03.635 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T01:33:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:33:03.671 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T01:33:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:33:03.718 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T01:33:03.742 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T01:33:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T01:33:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:33:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T01:33:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T01:33:03.848 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T01:33:03.899 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:33:03.935 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:33:03.964 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:33:03.992 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T01:33:03.997 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T01:33:04.054 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T01:33:04.092 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T01:33:04.103 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T01:33:04.133 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T01:33:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T01:33:04.227 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T01:33:04.238 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T01:33:04.267 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T01:33:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T01:33:04.307 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T01:33:04.333 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T01:33:04.337 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T01:33:04.358 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T01:33:04.372 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T01:33:04.415 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T01:33:04.431 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T01:33:04.448 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T01:33:04.476 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T01:33:04.544 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T01:33:04.559 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T01:33:04.597 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T01:33:04.599 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T01:33:04.626 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T01:33:04.654 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T01:33:04.661 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T01:33:04.700 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T01:33:04.714 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T01:33:04.732 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T01:33:04.762 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T01:33:04.797 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T01:33:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T01:33:04.830 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T01:33:04.833 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T01:33:04.872 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T01:33:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T01:33:04.897 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:33:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T01:33:04.906 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T01:33:04.936 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:33:04.975 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T01:33:04.993 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T01:33:05.020 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T01:33:05.038 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:33:05.044 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T01:33:05.076 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T01:33:05.093 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T01:33:05.123 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T01:33:05.207 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T01:33:05.222 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T01:33:05.227 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T01:33:05.246 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:33:05.269 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T01:33:05.269 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:33:05.270 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:05.270 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T01:33:05.270 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:33:05.270 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:33:05.270 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:33:05.280 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T01:33:05.313 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:33:05.324 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:33:05.421 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T01:33:05.469 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T01:33:05.806 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T01:33:05.884 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T01:33:05.922 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T01:33:05.965 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T01:33:05.984 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T01:33:06.003 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T01:33:06.034 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T01:33:06.044 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T01:33:06.067 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T01:33:06.165 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T01:33:06.199 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T01:33:06.217 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T01:33:06.256 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T01:33:06.311 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T01:33:06.321 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T01:33:06.353 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T01:33:06.363 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T01:33:06.384 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T01:33:06.390 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:33:06.413 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:33:06.413 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:06.413 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T01:33:06.414 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:33:06.414 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:33:06.414 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:33:06.419 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T01:33:06.443 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T01:33:06.466 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:33:06.488 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:33:06.489 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:06.489 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T01:33:06.489 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:33:06.490 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T01:33:06.526 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T01:33:06.532 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T01:33:06.550 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:33:06.559 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T01:33:06.569 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T01:33:06.573 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:33:06.573 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:06.573 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T01:33:06.573 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:33:06.573 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:33:06.573 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:33:06.612 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T01:33:06.637 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T01:33:06.747 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T01:33:06.792 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T01:33:06.822 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T01:33:06.863 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T01:33:06.876 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:33:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:33:06.899 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:06.899 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T01:33:06.899 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:33:06.899 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:33:06.899 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:33:06.958 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T01:33:06.989 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T01:33:07.029 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T01:33:07.055 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T01:33:07.080 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T01:33:07.153 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T01:33:07.280 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T01:33:07.288 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T01:33:07.312 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T01:33:07.340 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T01:33:07.374 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T01:33:07.400 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T01:33:07.413 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T01:33:07.446 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T01:33:07.475 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T01:33:07.510 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T01:33:07.543 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T01:33:07.584 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T01:33:07.619 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T01:33:07.706 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T01:33:07.749 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T01:33:07.781 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T01:33:07.827 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T01:33:07.925 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T01:33:07.974 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T01:33:08.625 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T01:33:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T01:33:08.707 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T01:33:08.735 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T01:33:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T01:33:08.768 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T01:33:08.771 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T01:33:08.869 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T01:33:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T01:33:09.009 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T01:33:09.040 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T01:33:09.082 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T01:33:09.123 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:33:09.213 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:33:09.218 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T01:33:09.244 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T01:33:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T01:33:09.280 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T01:33:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T01:33:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T01:33:09.776 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:33:09.847 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:33:09.877 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T01:33:09.972 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T01:33:10.220 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T01:33:11.396 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T01:33:11.437 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T01:33:11.441 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T01:33:11.477 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T01:33:11.510 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T01:33:11.526 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T01:33:11.537 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T01:33:11.582 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T01:33:11.612 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T01:33:11.719 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T01:33:11.775 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T01:33:11.805 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T01:33:11.924 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T01:33:11.967 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:33:12.058 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T01:33:12.088 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T01:33:12.124 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T01:33:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:33:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:33:12.434 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:33:12.477 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:33:12.496 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:33:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T01:33:12.628 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:33:12.700 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T01:33:12.730 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T01:33:12.748 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:33:12.780 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:33:12.872 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:33:12.872 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T01:33:12.872 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T01:33:12.872 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:33:12.906 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:33:13.331 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T01:33:13.415 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T01:33:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T01:33:13.546 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T01:33:13.579 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T01:33:13.579 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:33:13.672 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:33:13.713 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:33:13.713 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:33:13.811 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:33:13.878 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:33:14.956 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:33:14.993 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:33:15.530 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T01:33:15.572 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:33:15.668 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T01:33:15.669 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T01:33:15.669 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T01:33:15.669 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:33:15.699 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:33:19.686 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:33:19.686 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T01:33:19.686 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T01:33:19.686 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T01:33:19.686 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T01:33:19.686 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T01:33:19.686 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T01:33:19.687 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T01:33:19.687 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:33:19.687 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:33:19.687 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:33:19.687 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T01:33:19.687 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T01:33:19.687 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:33:19.687 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T01:33:19.687 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T01:33:19.687 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T01:33:19.687 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T01:33:19.687 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T01:33:19.687 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T01:33:19.687 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T01:33:19.688 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T01:33:19.688 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T01:33:19.688 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T01:33:19.688 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T01:33:19.688 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T01:33:19.688 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T01:33:19.688 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T01:33:19.688 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T01:33:19.688 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T01:33:19.688 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T01:33:19.700 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T01:33:19.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T01:33:19.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T01:33:19.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T01:33:19.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T01:33:19.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T01:33:19.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T01:33:19.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T01:33:19.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T01:33:19.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T01:33:19.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T01:33:19.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T01:33:19.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T01:33:19.701 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T01:33:19.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T01:33:19.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T01:33:19.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T01:33:19.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T01:33:19.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T01:33:19.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T01:33:19.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T01:33:19.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T01:33:19.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T01:33:19.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T01:33:19.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T01:33:19.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T01:33:19.702 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T01:33:19.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T01:33:19.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T01:33:19.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T01:33:19.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T01:33:19.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T01:33:19.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T01:33:19.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T01:33:19.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T01:33:19.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T01:33:19.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T01:33:19.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T01:33:19.703 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T01:33:19.704 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T01:33:19.704 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T01:33:19.704 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T01:33:19.704 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T01:33:19.704 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T01:33:19.704 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T01:33:19.704 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T01:33:19.704 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T01:33:19.704 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T01:33:19.704 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T01:33:19.704 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T01:33:19.704 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T01:33:19.704 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T01:33:19.704 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T01:33:19.705 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T01:33:19.705 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T01:33:19.705 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T01:33:19.705 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T01:33:19.705 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T01:33:19.705 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T01:33:19.705 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T01:33:19.705 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T01:33:19.705 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T01:33:19.705 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T01:33:19.705 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T01:33:19.705 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T01:33:19.705 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T01:33:19.706 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T01:33:19.706 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T01:33:19.706 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T01:33:19.706 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T01:33:19.706 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T01:33:19.706 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T01:33:19.706 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T01:33:19.706 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T01:33:19.706 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T01:33:19.706 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T01:33:19.706 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T01:33:19.706 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T01:33:19.706 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T01:33:19.706 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T01:33:19.707 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T01:33:19.707 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T01:33:19.707 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T01:33:19.707 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T01:33:19.707 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T01:33:19.707 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T01:33:19.707 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T01:33:19.707 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:33:19.707 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:33:19.707 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T01:33:19.707 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T01:33:19.707 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T01:33:19.707 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T01:33:19.708 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T01:33:19.708 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T01:33:19.708 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T01:33:19.708 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T01:33:19.708 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T01:33:19.708 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T01:33:19.708 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T01:33:19.708 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T01:33:19.708 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T01:33:19.708 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T01:33:19.708 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T01:33:19.708 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T01:33:19.708 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T01:33:19.708 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T01:33:19.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T01:33:19.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T01:33:19.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T01:33:19.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T01:33:19.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T01:33:19.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T01:33:19.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T01:33:19.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T01:33:19.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T01:33:19.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T01:33:19.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T01:33:19.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T01:33:19.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T01:33:19.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T01:33:19.710 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T01:33:19.710 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T01:33:19.710 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:33:19.710 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T01:33:21.821 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:33:21.821 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:33:21.821 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-04-25T01:33:21.822 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.822 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.822 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T01:33:21.822 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T01:33:21.822 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-04-25T01:33:21.823 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T01:33:21.823 INFO:teuthology.orchestra.run.smithi196.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T01:33:21.823 INFO:teuthology.orchestra.run.smithi196.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T01:33:21.823 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T01:33:21.823 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.823 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.824 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.824 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.824 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:21.824 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.824 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.825 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.825 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:21.825 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:21.825 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:21.825 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:21.825 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:21.826 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.826 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.826 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:21.826 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.826 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.826 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.827 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:21.827 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:21.827 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.827 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.827 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:21.827 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T01:33:21.827 INFO:teuthology.orchestra.run.smithi196.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T01:33:21.828 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T01:33:21.828 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T01:33:21.828 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T01:33:21.828 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T01:33:21.828 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T01:33:21.828 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T01:33:21.828 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T01:33:21.828 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T01:33:21.829 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T01:33:21.829 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.829 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.829 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.829 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T01:33:21.829 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T01:33:21.829 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T01:33:21.830 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T01:33:21.830 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T01:33:21.830 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.830 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.830 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T01:33:21.831 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.831 INFO:teuthology.orchestra.run.smithi196.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T01:33:21.831 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T01:33:21.831 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T01:33:21.831 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T01:33:21.831 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T01:33:21.831 INFO:teuthology.orchestra.run.smithi196.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T01:33:21.831 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T01:33:21.831 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T01:33:21.831 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T01:33:21.832 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T01:33:21.832 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T01:33:21.832 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T01:33:21.832 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T01:33:21.832 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T01:33:21.832 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T01:33:21.832 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T01:33:21.832 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T01:33:21.832 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T01:33:21.832 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T01:33:21.832 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T01:33:21.832 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T01:33:21.832 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T01:33:21.833 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T01:33:21.833 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.833 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.833 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.833 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T01:33:21.833 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T01:33:21.833 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T01:33:21.833 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T01:33:21.833 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T01:33:21.833 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T01:33:21.833 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T01:33:21.834 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T01:33:21.834 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T01:33:21.834 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T01:33:21.834 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T01:33:21.834 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T01:33:21.834 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T01:33:21.834 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T01:33:21.834 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T01:33:21.834 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T01:33:21.834 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T01:33:21.834 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T01:33:21.835 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T01:33:21.835 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T01:33:21.835 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T01:33:21.835 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T01:33:21.835 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T01:33:21.835 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T01:33:21.835 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T01:33:21.835 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T01:33:21.835 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T01:33:21.835 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T01:33:21.835 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T01:33:21.836 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T01:33:21.836 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T01:33:21.836 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T01:33:21.836 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T01:33:21.836 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T01:33:21.836 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T01:33:21.836 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T01:33:21.836 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T01:33:21.836 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T01:33:21.836 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T01:33:21.836 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T01:33:21.836 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.837 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.837 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T01:33:21.837 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T01:33:21.837 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T01:33:21.837 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.837 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T01:33:21.837 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T01:33:21.837 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T01:33:21.837 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T01:33:21.837 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T01:33:21.837 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T01:33:21.837 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T01:33:21.838 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T01:33:21.838 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T01:33:21.838 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T01:33:21.838 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T01:33:21.838 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T01:33:21.838 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T01:33:21.838 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T01:33:21.838 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T01:33:21.838 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.838 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.838 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:21.839 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T01:33:21.839 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T01:33:21.839 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T01:33:21.839 INFO:teuthology.orchestra.run.smithi196.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T01:33:21.839 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T01:33:21.839 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T01:33:21.839 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T01:33:21.840 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:33:21.840 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-04-25T01:33:22.232 DEBUG:teuthology.parallel:result is None 2024-04-25T01:33:24.543 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:33:24.543 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /sys 2024-04-25T01:33:24.543 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /proc 2024-04-25T01:33:24.544 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /mnt 2024-04-25T01:33:24.544 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /var/tmp 2024-04-25T01:33:24.544 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /home 2024-04-25T01:33:24.544 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /root 2024-04-25T01:33:24.544 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /tmp 2024-04-25T01:33:24.544 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:33:24.713 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:33:24.729 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:33:24.729 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:24.729 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T01:33:24.729 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:33:26.200 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:33:26.226 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:33:26.226 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:26.226 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T01:33:26.226 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:33:26.226 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:33:26.226 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:33:26.459 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:33:26.482 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:33:26.482 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:26.482 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T01:33:26.482 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:33:26.482 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:33:26.482 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:33:26.926 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:33:26.949 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:33:26.949 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:26.949 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T01:33:26.949 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:33:26.949 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:33:26.949 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:33:27.057 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T01:33:27.113 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T01:33:27.169 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T01:33:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T01:33:27.272 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /sys 2024-04-25T01:33:27.272 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /proc 2024-04-25T01:33:27.272 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /mnt 2024-04-25T01:33:27.272 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /var/tmp 2024-04-25T01:33:27.272 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /home 2024-04-25T01:33:27.273 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /root 2024-04-25T01:33:27.273 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /tmp 2024-04-25T01:33:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:33:27.354 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T01:33:27.403 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T01:33:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:33:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T01:33:27.451 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:27.451 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T01:33:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:33:27.480 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T01:33:27.570 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:33:27.724 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:33:27.769 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T01:33:27.802 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T01:33:27.829 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T01:33:28.240 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:33:28.270 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:33:28.301 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T01:33:28.364 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T01:33:28.401 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:33:28.924 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:33:28.949 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T01:33:28.949 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:28.950 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T01:33:28.950 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:33:28.950 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:33:28.950 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:33:29.178 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:33:29.200 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T01:33:29.200 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:29.201 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T01:33:29.201 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:33:29.201 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:33:29.201 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:33:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:33:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:33:29.645 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:33:29.667 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T01:33:29.668 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:29.668 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T01:33:29.668 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:33:29.668 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:33:29.668 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:33:29.775 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T01:33:29.823 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T01:33:29.880 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T01:33:30.064 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T01:33:30.112 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T01:33:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T01:33:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:33:30.427 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:33:30.432 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:33:30.462 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:33:30.463 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T01:33:30.495 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T01:33:30.522 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T01:33:30.566 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:33:30.727 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T01:33:30.756 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:33:30.787 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:33:30.787 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:30.788 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T01:33:30.788 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:33:30.788 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:33:30.788 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:33:30.849 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:33:30.849 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:33:30.942 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T01:33:30.946 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:33:30.972 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T01:33:30.988 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T01:33:31.003 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T01:33:31.064 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T01:33:31.103 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:33:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T01:33:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:33:31.932 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:33:31.932 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:31.933 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T01:33:31.933 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:33:31.933 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:33:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:33:31.977 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:33:32.002 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:33:32.002 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:32.002 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T01:33:32.002 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:33:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:33:32.075 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:33:32.076 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:32.076 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T01:33:32.076 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:33:32.076 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:33:32.076 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:33:32.134 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T01:33:32.171 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:33:32.378 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:33:32.401 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:33:32.401 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:32.402 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T01:33:32.402 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:33:32.402 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:33:32.402 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:33:33.140 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T01:33:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:33:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T01:33:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T01:33:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:33:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T01:33:33.480 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:33.480 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T01:33:33.480 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:33:33.481 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:33:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:33:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:33:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T01:33:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T01:33:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T01:33:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T01:33:34.627 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:33:34.650 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T01:33:34.650 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:34.650 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T01:33:34.650 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:33:34.650 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:33:34.650 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:33:34.695 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:33:34.716 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T01:33:34.717 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:34.717 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T01:33:34.717 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:33:34.762 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:33:34.783 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T01:33:34.784 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:34.784 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T01:33:34.784 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:33:34.784 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:33:34.784 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:33:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:33:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T01:33:35.135 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:33:35.135 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T01:33:35.135 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:33:35.135 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:33:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:33:36.933 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T01:33:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T01:33:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T01:33:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T01:33:37.075 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T01:33:37.256 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T01:33:37.305 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T01:33:37.335 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T01:33:37.966 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:33:38.010 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:33:38.030 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:33:38.866 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T01:33:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T01:33:38.990 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T01:33:39.072 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T01:33:39.112 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T01:33:39.112 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:33:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:33:39.229 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:33:39.229 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:33:39.317 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:33:39.386 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:33:39.616 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T01:33:39.658 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T01:33:39.706 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T01:33:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T01:33:39.766 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T01:33:39.945 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T01:33:39.988 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T01:33:40.017 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T01:33:40.636 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:33:40.674 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:33:40.694 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T01:33:41.512 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T01:33:41.595 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T01:33:41.637 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T01:33:41.717 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T01:33:41.750 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T01:33:41.750 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:33:41.834 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T01:33:41.859 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:33:41.859 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:33:41.961 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T01:33:42.032 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:33:45.098 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:33:45.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T01:33:45.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T01:33:45.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T01:33:45.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T01:33:45.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T01:33:45.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T01:33:45.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T01:33:45.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:33:45.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:33:45.100 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:33:45.100 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T01:33:45.100 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T01:33:45.100 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:33:45.100 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T01:33:45.100 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T01:33:45.101 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T01:33:45.101 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T01:33:45.101 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T01:33:45.101 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T01:33:45.101 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T01:33:45.101 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T01:33:45.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T01:33:45.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T01:33:45.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T01:33:45.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T01:33:45.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T01:33:45.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T01:33:45.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T01:33:45.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T01:33:45.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T01:33:45.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T01:33:45.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T01:33:45.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T01:33:45.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T01:33:45.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T01:33:45.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T01:33:45.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T01:33:45.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T01:33:45.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T01:33:45.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T01:33:45.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T01:33:45.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T01:33:45.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T01:33:45.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T01:33:45.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T01:33:45.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T01:33:45.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T01:33:45.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T01:33:45.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T01:33:45.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T01:33:45.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T01:33:45.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T01:33:45.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T01:33:45.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T01:33:45.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T01:33:45.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T01:33:45.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T01:33:45.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T01:33:45.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T01:33:45.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T01:33:45.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T01:33:45.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T01:33:45.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T01:33:45.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T01:33:45.120 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T01:33:45.120 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T01:33:45.120 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T01:33:45.120 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T01:33:45.120 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T01:33:45.120 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T01:33:45.120 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T01:33:45.120 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T01:33:45.120 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T01:33:45.120 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T01:33:45.120 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T01:33:45.120 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T01:33:45.120 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T01:33:45.120 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T01:33:45.121 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T01:33:45.121 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T01:33:45.121 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T01:33:45.121 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T01:33:45.121 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T01:33:45.121 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T01:33:45.121 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T01:33:45.121 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T01:33:45.121 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T01:33:45.121 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T01:33:45.121 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T01:33:45.121 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T01:33:45.121 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T01:33:45.122 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T01:33:45.122 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T01:33:45.122 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T01:33:45.122 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T01:33:45.122 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T01:33:45.122 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T01:33:45.122 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T01:33:45.122 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T01:33:45.122 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T01:33:45.122 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T01:33:45.122 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T01:33:45.122 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T01:33:45.122 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T01:33:45.123 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T01:33:45.123 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T01:33:45.123 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T01:33:45.123 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T01:33:45.123 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T01:33:45.123 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T01:33:45.123 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T01:33:45.123 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T01:33:45.123 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T01:33:45.123 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T01:33:45.124 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T01:33:45.124 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T01:33:45.124 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T01:33:45.124 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:33:45.124 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:33:45.124 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T01:33:45.124 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T01:33:45.124 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T01:33:45.124 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T01:33:45.124 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T01:33:45.124 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T01:33:45.124 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T01:33:45.124 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T01:33:45.124 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T01:33:45.125 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T01:33:45.125 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T01:33:45.125 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T01:33:45.125 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T01:33:45.125 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T01:33:45.125 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T01:33:45.125 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T01:33:45.125 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T01:33:45.125 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T01:33:45.125 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T01:33:45.125 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T01:33:45.125 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T01:33:45.125 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T01:33:45.125 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T01:33:45.126 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T01:33:45.126 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T01:33:45.126 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T01:33:45.126 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T01:33:45.126 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T01:33:45.126 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T01:33:45.126 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T01:33:45.126 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T01:33:45.126 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T01:33:45.126 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T01:33:45.126 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T01:33:45.126 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:33:45.126 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T01:33:47.242 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:33:47.242 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:33:47.243 INFO:teuthology.orchestra.run.smithi152.stdout:Upgraded: 2024-04-25T01:33:47.243 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.243 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.243 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T01:33:47.243 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T01:33:47.243 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-04-25T01:33:47.243 INFO:teuthology.orchestra.run.smithi152.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T01:33:47.243 INFO:teuthology.orchestra.run.smithi152.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T01:33:47.244 INFO:teuthology.orchestra.run.smithi152.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T01:33:47.244 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T01:33:47.244 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.244 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.244 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.245 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.245 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:47.245 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.245 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.245 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.245 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.247 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.247 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:47.247 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:47.247 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.247 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.247 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:47.247 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T01:33:47.247 INFO:teuthology.orchestra.run.smithi152.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T01:33:47.247 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T01:33:47.248 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T01:33:47.248 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T01:33:47.248 INFO:teuthology.orchestra.run.smithi152.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T01:33:47.248 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T01:33:47.248 INFO:teuthology.orchestra.run.smithi152.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T01:33:47.248 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T01:33:47.248 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T01:33:47.248 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T01:33:47.248 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.248 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.248 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.249 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T01:33:47.249 INFO:teuthology.orchestra.run.smithi152.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T01:33:47.249 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T01:33:47.249 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T01:33:47.249 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T01:33:47.249 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.249 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.249 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T01:33:47.249 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.249 INFO:teuthology.orchestra.run.smithi152.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T01:33:47.249 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T01:33:47.249 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T01:33:47.250 INFO:teuthology.orchestra.run.smithi152.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T01:33:47.250 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T01:33:47.250 INFO:teuthology.orchestra.run.smithi152.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T01:33:47.250 INFO:teuthology.orchestra.run.smithi152.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T01:33:47.250 INFO:teuthology.orchestra.run.smithi152.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T01:33:47.250 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T01:33:47.250 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T01:33:47.250 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T01:33:47.250 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T01:33:47.250 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T01:33:47.250 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T01:33:47.250 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T01:33:47.250 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T01:33:47.251 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T01:33:47.251 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T01:33:47.251 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T01:33:47.251 INFO:teuthology.orchestra.run.smithi152.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T01:33:47.251 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T01:33:47.251 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T01:33:47.251 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T01:33:47.251 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.251 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.251 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.251 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T01:33:47.251 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T01:33:47.251 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T01:33:47.252 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T01:33:47.252 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T01:33:47.252 INFO:teuthology.orchestra.run.smithi152.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T01:33:47.252 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T01:33:47.252 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T01:33:47.252 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T01:33:47.252 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T01:33:47.252 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T01:33:47.252 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T01:33:47.252 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T01:33:47.252 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T01:33:47.252 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T01:33:47.252 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T01:33:47.252 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T01:33:47.253 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T01:33:47.253 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T01:33:47.253 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T01:33:47.253 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T01:33:47.253 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T01:33:47.253 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T01:33:47.253 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T01:33:47.253 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T01:33:47.253 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T01:33:47.253 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T01:33:47.253 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T01:33:47.253 INFO:teuthology.orchestra.run.smithi152.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T01:33:47.253 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T01:33:47.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T01:33:47.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T01:33:47.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T01:33:47.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T01:33:47.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T01:33:47.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T01:33:47.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T01:33:47.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T01:33:47.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T01:33:47.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T01:33:47.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.254 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T01:33:47.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T01:33:47.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T01:33:47.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T01:33:47.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T01:33:47.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T01:33:47.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T01:33:47.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T01:33:47.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T01:33:47.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T01:33:47.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T01:33:47.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T01:33:47.255 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T01:33:47.256 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T01:33:47.256 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T01:33:47.256 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T01:33:47.256 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T01:33:47.256 INFO:teuthology.orchestra.run.smithi152.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T01:33:47.256 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.256 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.256 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:47.256 INFO:teuthology.orchestra.run.smithi152.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T01:33:47.257 INFO:teuthology.orchestra.run.smithi152.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T01:33:47.257 INFO:teuthology.orchestra.run.smithi152.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T01:33:47.257 INFO:teuthology.orchestra.run.smithi152.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T01:33:47.257 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T01:33:47.257 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T01:33:47.257 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T01:33:47.257 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T01:33:47.257 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-04-25T01:33:47.614 DEBUG:teuthology.parallel:result is None 2024-04-25T01:33:47.819 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:33:47.819 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T01:33:47.819 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T01:33:47.819 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T01:33:47.819 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T01:33:47.819 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T01:33:47.819 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T01:33:47.819 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T01:33:47.819 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T01:33:47.820 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T01:33:47.820 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T01:33:47.820 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T01:33:47.820 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T01:33:47.820 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T01:33:47.820 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T01:33:47.820 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T01:33:47.820 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T01:33:47.821 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T01:33:47.821 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T01:33:47.821 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T01:33:47.821 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T01:33:47.821 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T01:33:47.821 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T01:33:47.821 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T01:33:47.821 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T01:33:47.822 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T01:33:47.822 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T01:33:47.822 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T01:33:47.822 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T01:33:47.822 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T01:33:47.835 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T01:33:47.835 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T01:33:47.836 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T01:33:47.836 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T01:33:47.836 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T01:33:47.836 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T01:33:47.836 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T01:33:47.836 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T01:33:47.836 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T01:33:47.836 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T01:33:47.836 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T01:33:47.837 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T01:33:47.837 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T01:33:47.837 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T01:33:47.837 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T01:33:47.837 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T01:33:47.837 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T01:33:47.837 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T01:33:47.837 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T01:33:47.837 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T01:33:47.837 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T01:33:47.838 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T01:33:47.838 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T01:33:47.838 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T01:33:47.838 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T01:33:47.838 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T01:33:47.838 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T01:33:47.838 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T01:33:47.838 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T01:33:47.838 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T01:33:47.838 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T01:33:47.839 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T01:33:47.839 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T01:33:47.839 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T01:33:47.839 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T01:33:47.839 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T01:33:47.839 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T01:33:47.839 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T01:33:47.839 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T01:33:47.839 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T01:33:47.839 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T01:33:47.839 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T01:33:47.839 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T01:33:47.839 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T01:33:47.840 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T01:33:47.840 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T01:33:47.840 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T01:33:47.840 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T01:33:47.840 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T01:33:47.840 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T01:33:47.840 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T01:33:47.840 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T01:33:47.840 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T01:33:47.841 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T01:33:47.841 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T01:33:47.841 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T01:33:47.841 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T01:33:47.841 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T01:33:47.841 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T01:33:47.841 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T01:33:47.841 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T01:33:47.841 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T01:33:47.841 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T01:33:47.842 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T01:33:47.842 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T01:33:47.842 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T01:33:47.842 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T01:33:47.842 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T01:33:47.842 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T01:33:47.842 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T01:33:47.842 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T01:33:47.842 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T01:33:47.842 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T01:33:47.842 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T01:33:47.842 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T01:33:47.842 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T01:33:47.843 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T01:33:47.843 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T01:33:47.843 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T01:33:47.843 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T01:33:47.843 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T01:33:47.843 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T01:33:47.843 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T01:33:47.843 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T01:33:47.843 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T01:33:47.843 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T01:33:47.843 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T01:33:47.843 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T01:33:47.843 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T01:33:47.843 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T01:33:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T01:33:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T01:33:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T01:33:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T01:33:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T01:33:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T01:33:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T01:33:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T01:33:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T01:33:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T01:33:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T01:33:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T01:33:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T01:33:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T01:33:47.845 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T01:33:47.845 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T01:33:47.845 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T01:33:47.845 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T01:33:47.845 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T01:33:47.845 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T01:33:47.845 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T01:33:47.845 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T01:33:47.845 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T01:33:47.845 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T01:33:47.846 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T01:33:47.846 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T01:33:47.846 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T01:33:47.846 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T01:33:47.846 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T01:33:47.846 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T01:33:47.846 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T01:33:47.846 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T01:33:47.846 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T01:33:47.846 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T01:33:47.846 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T01:33:47.846 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T01:33:49.882 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T01:33:49.883 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:33:49.883 INFO:teuthology.orchestra.run.smithi078.stdout:Upgraded: 2024-04-25T01:33:49.883 INFO:teuthology.orchestra.run.smithi078.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.883 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.883 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T01:33:49.883 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T01:33:49.883 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-25T01:33:49.884 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T01:33:49.884 INFO:teuthology.orchestra.run.smithi078.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T01:33:49.884 INFO:teuthology.orchestra.run.smithi078.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T01:33:49.884 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T01:33:49.884 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.884 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.885 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.885 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.885 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:49.885 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.885 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.885 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.886 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:49.886 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:49.886 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:49.886 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:49.886 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:49.887 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.887 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.887 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:49.887 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.887 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.887 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.888 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:49.888 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:49.888 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.888 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.888 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T01:33:49.888 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T01:33:49.889 INFO:teuthology.orchestra.run.smithi078.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T01:33:49.889 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T01:33:49.889 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T01:33:49.889 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T01:33:49.889 INFO:teuthology.orchestra.run.smithi078.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T01:33:49.889 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T01:33:49.890 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T01:33:49.890 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T01:33:49.890 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T01:33:49.890 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T01:33:49.890 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.891 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.891 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.891 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T01:33:49.891 INFO:teuthology.orchestra.run.smithi078.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T01:33:49.891 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T01:33:49.891 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T01:33:49.892 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T01:33:49.892 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.892 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.892 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T01:33:49.892 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.893 INFO:teuthology.orchestra.run.smithi078.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T01:33:49.893 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T01:33:49.893 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T01:33:49.893 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T01:33:49.893 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T01:33:49.893 INFO:teuthology.orchestra.run.smithi078.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T01:33:49.893 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T01:33:49.893 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T01:33:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T01:33:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T01:33:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T01:33:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T01:33:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T01:33:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T01:33:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T01:33:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T01:33:49.895 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T01:33:49.895 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T01:33:49.895 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T01:33:49.895 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T01:33:49.895 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T01:33:49.895 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T01:33:49.895 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T01:33:49.895 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.896 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.896 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.896 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T01:33:49.896 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T01:33:49.896 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T01:33:49.896 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T01:33:49.897 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T01:33:49.897 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T01:33:49.897 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T01:33:49.897 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T01:33:49.897 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T01:33:49.897 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T01:33:49.898 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T01:33:49.898 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T01:33:49.898 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T01:33:49.898 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T01:33:49.898 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T01:33:49.898 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T01:33:49.899 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T01:33:49.899 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T01:33:49.899 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T01:33:49.899 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T01:33:49.899 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T01:33:49.899 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T01:33:49.900 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T01:33:49.900 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T01:33:49.900 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T01:33:49.900 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T01:33:49.900 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T01:33:49.900 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T01:33:49.901 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T01:33:49.901 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T01:33:49.901 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T01:33:49.901 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T01:33:49.901 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T01:33:49.901 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T01:33:49.902 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T01:33:49.902 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T01:33:49.902 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T01:33:49.902 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T01:33:49.902 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T01:33:49.903 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T01:33:49.903 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.903 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.903 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T01:33:49.903 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T01:33:49.903 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T01:33:49.903 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.903 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T01:33:49.904 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T01:33:49.904 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T01:33:49.904 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T01:33:49.904 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T01:33:49.904 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T01:33:49.904 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T01:33:49.904 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T01:33:49.904 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T01:33:49.904 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T01:33:49.904 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T01:33:49.904 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T01:33:49.904 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T01:33:49.904 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T01:33:49.905 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T01:33:49.905 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.905 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.905 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T01:33:49.905 INFO:teuthology.orchestra.run.smithi078.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T01:33:49.905 INFO:teuthology.orchestra.run.smithi078.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T01:33:49.905 INFO:teuthology.orchestra.run.smithi078.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T01:33:49.905 INFO:teuthology.orchestra.run.smithi078.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T01:33:49.905 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T01:33:49.905 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T01:33:49.905 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T01:33:49.905 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:33:49.905 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-25T01:33:50.268 DEBUG:teuthology.parallel:result is None 2024-04-25T01:33:50.268 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:33:50.418 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T01:33:50.442 INFO:teuthology.orchestra.run.smithi078.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T01:33:50.442 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T01:33:50.443 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T01:33:50.445 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:33:50.666 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T01:33:50.691 INFO:teuthology.orchestra.run.smithi152.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T01:33:50.692 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T01:33:50.692 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T01:33:50.695 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:33:50.923 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T01:33:50.948 INFO:teuthology.orchestra.run.smithi196.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T01:33:50.949 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T01:33:50.949 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T01:33:50.951 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T01:33:50.951 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:33:50.952 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T01:33:50.978 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T01:33:50.979 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T01:33:51.009 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T01:33:51.009 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T01:33:51.039 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T01:33:51.039 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:33:51.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T01:33:51.064 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T01:33:51.128 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T01:33:51.128 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T01:33:51.163 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T01:33:51.229 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T01:33:51.229 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T01:33:51.253 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T01:33:51.318 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T01:33:51.318 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:33:51.318 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T01:33:51.342 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T01:33:51.406 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T01:33:51.406 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T01:33:51.435 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T01:33:51.500 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T01:33:51.500 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T01:33:51.525 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T01:33:51.589 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T01:33:51.589 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:33:51.589 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T01:33:51.613 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T01:33:51.676 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T01:33:51.676 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T01:33:51.701 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T01:33:51.765 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T01:33:51.766 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T01:33:51.789 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T01:33:51.852 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T01:33:51.960 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-25T01:33:51.960 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:33:51.960 INFO:tasks.cephadm:Cluster fsid is df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:33:51.960 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T01:33:51.961 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.78', 'mon.b': '172.21.15.152', 'mon.c': '172.21.15.196'} 2024-04-25T01:33:51.961 INFO:tasks.cephadm:First mon is mon.a on smithi078 2024-04-25T01:33:51.961 INFO:tasks.cephadm:First mgr is x 2024-04-25T01:33:51.961 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T01:33:51.961 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname $(hostname -s) 2024-04-25T01:33:51.985 DEBUG:teuthology.orchestra.run.smithi152:> sudo hostname $(hostname -s) 2024-04-25T01:33:52.010 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-04-25T01:33:52.033 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T01:33:52.034 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:33:52.177 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54987/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 22:37:46.422067', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54981/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 17:00:00.872117', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T01:33:52.325 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 d2c45d977142e024a59b68a8f1dce8f0b9840551 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:33:52.326 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm 2024-04-25T01:33:52.327 DEBUG:teuthology.orchestra.run.smithi078:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T01:33:53.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 01:33 /home/ubuntu/cephtest/cephadm 2024-04-25T01:33:53.575 DEBUG:teuthology.orchestra.run.smithi152:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T01:33:54.557 INFO:teuthology.orchestra.run.smithi152.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 01:33 /home/ubuntu/cephtest/cephadm 2024-04-25T01:33:54.558 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T01:33:55.396 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 01:33 /home/ubuntu/cephtest/cephadm 2024-04-25T01:33:55.397 DEBUG:teuthology.orchestra.run.smithi078:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T01:33:55.416 DEBUG:teuthology.orchestra.run.smithi152:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T01:33:55.434 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T01:33:55.469 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 on all hosts... 2024-04-25T01:33:55.469 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T01:33:55.472 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T01:33:55.477 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T01:33:57.031 INFO:teuthology.orchestra.run.smithi078.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T01:33:57.119 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T01:33:57.139 INFO:teuthology.orchestra.run.smithi152.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T01:35:06.041 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:35:06.041 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:35:06.041 INFO:teuthology.orchestra.run.smithi078.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T01:35:06.041 INFO:teuthology.orchestra.run.smithi078.stdout: "repo_digests": [ 2024-04-25T01:35:06.041 INFO:teuthology.orchestra.run.smithi078.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T01:35:06.041 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T01:35:06.042 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:35:08.565 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-25T01:35:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:35:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T01:35:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: "repo_digests": [ 2024-04-25T01:35:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T01:35:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-25T01:35:08.566 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-25T01:35:13.456 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-25T01:35:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:35:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T01:35:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-04-25T01:35:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T01:35:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-25T01:35:13.457 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-25T01:35:13.482 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph 2024-04-25T01:35:13.514 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph 2024-04-25T01:35:13.546 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-04-25T01:35:13.574 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /etc/ceph 2024-04-25T01:35:13.601 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 777 /etc/ceph 2024-04-25T01:35:13.628 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-04-25T01:35:13.653 INFO:tasks.cephadm:Writing seed config... 2024-04-25T01:35:13.654 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T01:35:13.654 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T01:35:13.654 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T01:35:13.654 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T01:35:13.654 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T01:35:13.654 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T01:35:13.654 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T01:35:13.654 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T01:35:13.654 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T01:35:13.655 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-25T01:35:13.656 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T01:35:13.657 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T01:35:13.657 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T01:35:13.657 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T01:35:13.657 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T01:35:13.657 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T01:35:13.657 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T01:35:13.657 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-25T01:35:13.658 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:35:13.658 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T01:35:13.675 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = df36bea8-02a3-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T01:35:13.676 DEBUG:teuthology.orchestra.run.smithi078:mon.a> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T01:35:13.718 DEBUG:teuthology.orchestra.run.smithi078:mgr.x> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T01:35:13.761 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T01:35:13.761 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 -v bootstrap --fsid df36bea8-02a3-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.78 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:35:13.953 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:35:13.953 INFO:teuthology.orchestra.run.smithi078.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551', '-v', 'bootstrap', '--fsid', 'df36bea8-02a3-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.78', '--skip-admin-label'] 2024-04-25T01:35:13.954 INFO:teuthology.orchestra.run.smithi078.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T01:35:13.954 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying podman|docker is present... 2024-04-25T01:35:13.980 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T01:35:13.980 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying lvm2 is present... 2024-04-25T01:35:13.981 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying time synchronization is in place... 2024-04-25T01:35:13.988 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T01:35:13.988 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T01:35:13.994 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T01:35:13.994 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-04-25T01:35:14.000 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout enabled 2024-04-25T01:35:14.006 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout active 2024-04-25T01:35:14.006 INFO:teuthology.orchestra.run.smithi078.stdout:Unit chronyd.service is enabled and running 2024-04-25T01:35:14.006 INFO:teuthology.orchestra.run.smithi078.stdout:Repeating the final host check... 2024-04-25T01:35:14.029 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T01:35:14.030 INFO:teuthology.orchestra.run.smithi078.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T01:35:14.030 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl is present 2024-04-25T01:35:14.030 INFO:teuthology.orchestra.run.smithi078.stdout:lvcreate is present 2024-04-25T01:35:14.036 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T01:35:14.036 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T01:35:14.042 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T01:35:14.042 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-04-25T01:35:14.047 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout enabled 2024-04-25T01:35:14.054 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout active 2024-04-25T01:35:14.054 INFO:teuthology.orchestra.run.smithi078.stdout:Unit chronyd.service is enabled and running 2024-04-25T01:35:14.054 INFO:teuthology.orchestra.run.smithi078.stdout:Host looks OK 2024-04-25T01:35:14.054 INFO:teuthology.orchestra.run.smithi078.stdout:Cluster fsid: df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:35:14.054 INFO:teuthology.orchestra.run.smithi078.stdout:Acquiring lock 139642756129696 on /run/cephadm/df36bea8-02a3-11ef-bc93-c7b262605968.lock 2024-04-25T01:35:14.054 INFO:teuthology.orchestra.run.smithi078.stdout:Lock 139642756129696 acquired on /run/cephadm/df36bea8-02a3-11ef-bc93-c7b262605968.lock 2024-04-25T01:35:14.054 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying IP 172.21.15.78 port 3300 ... 2024-04-25T01:35:14.055 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying IP 172.21.15.78 port 6789 ... 2024-04-25T01:35:14.055 INFO:teuthology.orchestra.run.smithi078.stdout:Base mon IP(s) is [172.21.15.78:3300, 172.21.15.78:6789], mon addrv is [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-04-25T01:35:14.057 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.78 metric 100 2024-04-25T01:35:14.057 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.78 metric 100 2024-04-25T01:35:14.060 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T01:35:14.060 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-25T01:35:14.060 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-25T01:35:14.063 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T01:35:14.063 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T01:35:14.063 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T01:35:14.063 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T01:35:14.063 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout inet6 fe80::7539:9f76:2904:c153/64 scope link noprefixroute 2024-04-25T01:35:14.064 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T01:35:14.064 INFO:teuthology.orchestra.run.smithi078.stdout:Mon IP `172.21.15.78` is in CIDR network `172.21.0.0/20` 2024-04-25T01:35:14.064 INFO:teuthology.orchestra.run.smithi078.stdout:Mon IP `172.21.15.78` is in CIDR network `172.21.0.0/20` 2024-04-25T01:35:14.064 INFO:teuthology.orchestra.run.smithi078.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T01:35:14.065 INFO:teuthology.orchestra.run.smithi078.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T01:35:14.065 INFO:teuthology.orchestra.run.smithi078.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T01:35:14.559 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-25T01:35:14.559 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T01:35:14.559 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T01:35:14.559 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T01:35:14.559 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying blob sha256:58a99251f41e8a467bafa0ec849428709568906ca732f10972304f37056be6c5 2024-04-25T01:35:14.559 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying config sha256:f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-25T01:35:14.559 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T01:35:15.474 INFO:teuthology.orchestra.run.smithi078.stdout:ceph: stdout ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-25T01:35:15.475 INFO:teuthology.orchestra.run.smithi078.stdout:Ceph version: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-25T01:35:15.475 INFO:teuthology.orchestra.run.smithi078.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T01:35:16.331 INFO:teuthology.orchestra.run.smithi078.stdout:stat: stdout 167 167 2024-04-25T01:35:16.331 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial keys... 2024-04-25T01:35:17.371 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQBUsylmZHQXKRAA6kElItnNPI9RwuhdZQzv7A== 2024-04-25T01:35:18.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQBVsylmJwORKRAAn/6TTAa6iFi8QStYT9dfOg== 2024-04-25T01:35:19.193 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQBWsylmjrryJhAAunQMnDjg6wR3IAHEHMCwRQ== 2024-04-25T01:35:19.194 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial monmap... 2024-04-25T01:35:20.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T01:35:20.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T01:35:20.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:35:20.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T01:35:20.092 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool for a [v2:172.21.15.78:3300,v1:172.21.15.78:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T01:35:20.092 INFO:teuthology.orchestra.run.smithi078.stdout:setting min_mon_release = quincy 2024-04-25T01:35:20.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: set fsid to df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:35:20.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T01:35:20.092 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:35:20.092 INFO:teuthology.orchestra.run.smithi078.stdout:Creating mon... 2024-04-25T01:35:21.099 INFO:teuthology.orchestra.run.smithi078.stdout:create mon.a on 2024-04-25T01:35:21.319 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T01:35:21.460 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T01:35:21.639 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-df36bea8-02a3-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-df36bea8-02a3-11ef-bc93-c7b262605968.target. 2024-04-25T01:35:21.639 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-df36bea8-02a3-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-df36bea8-02a3-11ef-bc93-c7b262605968.target. 2024-04-25T01:35:21.859 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.a 2024-04-25T01:35:21.859 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to reset failed state of unit ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T01:35:22.024 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-df36bea8-02a3-11ef-bc93-c7b262605968.target.wants/ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-df36bea8-02a3-11ef-bc93-c7b262605968@.service. 2024-04-25T01:35:22.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:22 smithi078 ceph-mon[29089]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T01:35:22.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:22 smithi078 bash[29063]: 4fe595d235eb62c998b3eba8a175ecb311227709994cb201a3726497742f2379 2024-04-25T01:35:22.560 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T01:35:22.560 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-04-25T01:35:22.567 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T01:35:22.567 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-04-25T01:35:22.567 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-04-25T01:35:22.567 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T01:35:22.568 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mon to start... 2024-04-25T01:35:22.568 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mon... 2024-04-25T01:35:22.858 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:22 smithi078 systemd[1]: Started Ceph mon.a for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:35:23.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:23 smithi078 ceph-mon[29089]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T01:35:23.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:23 smithi078 ceph-mon[29089]: monmap epoch 1 2024-04-25T01:35:23.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:23 smithi078 ceph-mon[29089]: fsid df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:35:23.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:23 smithi078 ceph-mon[29089]: last_changed 2024-04-25T01:35:19.558079+0000 2024-04-25T01:35:23.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:23 smithi078 ceph-mon[29089]: created 2024-04-25T01:35:19.558079+0000 2024-04-25T01:35:23.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:23 smithi078 ceph-mon[29089]: min_mon_release 19 (squid) 2024-04-25T01:35:23.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:23 smithi078 ceph-mon[29089]: election_strategy: 1 2024-04-25T01:35:23.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:23 smithi078 ceph-mon[29089]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T01:35:23.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:23 smithi078 ceph-mon[29089]: fsmap 2024-04-25T01:35:23.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:23 smithi078 ceph-mon[29089]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T01:35:23.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:23 smithi078 ceph-mon[29089]: mgrmap e1: no daemons active 2024-04-25T01:35:23.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:23 smithi078 ceph-mon[29089]: from='client.? 172.21.15.78:0/101060322' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T01:35:23.616 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T01:35:23.623 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout id: df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:35:23.623 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T01:35:23.623 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T01:35:23.623 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout services: 2024-04-25T01:35:23.623 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.531594s) 2024-04-25T01:35:23.623 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T01:35:23.623 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T01:35:23.623 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T01:35:23.623 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout data: 2024-04-25T01:35:23.623 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T01:35:23.623 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T01:35:23.624 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T01:35:23.624 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T01:35:23.624 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T01:35:23.624 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.980+0000 7f44bb90c700 1 Processor -- start 2024-04-25T01:35:23.624 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.981+0000 7f44bb90c700 1 -- start start 2024-04-25T01:35:23.624 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.981+0000 7f44bb90c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4105fd0 0x7f44b41063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:23.624 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.981+0000 7f44bb90c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f44b41069c0 con 0x7f44b4105fd0 2024-04-25T01:35:23.624 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.981+0000 7f44b96a8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4105fd0 0x7f44b41063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:23.624 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.981+0000 7f44b96a8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4105fd0 0x7f44b41063f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37568/0 (socket says 172.21.15.78:37568) 2024-04-25T01:35:23.624 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.981+0000 7f44b96a8700 1 -- 172.21.15.78:0/441245529 learned_addr learned my addr 172.21.15.78:0/441245529 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:23.624 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.982+0000 7f44b96a8700 1 -- 172.21.15.78:0/441245529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b4108f60 con 0x7f44b4105fd0 2024-04-25T01:35:23.624 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.982+0000 7f44b96a8700 1 --2- 172.21.15.78:0/441245529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4105fd0 0x7f44b41063f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f44b00098a0 tx=0x7f44b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4f2b3f5204af9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:23.624 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.982+0000 7f44abfff700 1 -- 172.21.15.78:0/441245529 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44b0004160 con 0x7f44b4105fd0 2024-04-25T01:35:23.625 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.982+0000 7f44abfff700 1 -- 172.21.15.78:0/441245529 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44b0004300 con 0x7f44b4105fd0 2024-04-25T01:35:23.625 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.982+0000 7f44abfff700 1 -- 172.21.15.78:0/441245529 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44b00036a0 con 0x7f44b4105fd0 2024-04-25T01:35:23.625 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.982+0000 7f44bb90c700 1 -- 172.21.15.78:0/441245529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4105fd0 msgr2=0x7f44b41063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:23.625 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.982+0000 7f44bb90c700 1 --2- 172.21.15.78:0/441245529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4105fd0 0x7f44b41063f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f44b00098a0 tx=0x7f44b0009bb0 comp rx=0 tx=0).stop 2024-04-25T01:35:23.625 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.983+0000 7f44bb90c700 1 -- 172.21.15.78:0/441245529 shutdown_connections 2024-04-25T01:35:23.625 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.983+0000 7f44bb90c700 1 --2- 172.21.15.78:0/441245529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4105fd0 0x7f44b41063f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:23.625 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.983+0000 7f44bb90c700 1 -- 172.21.15.78:0/441245529 >> 172.21.15.78:0/441245529 conn(0x7f44b4101900 msgr2=0x7f44b4103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:23.625 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.983+0000 7f44bb90c700 1 -- 172.21.15.78:0/441245529 shutdown_connections 2024-04-25T01:35:23.625 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.983+0000 7f44bb90c700 1 -- 172.21.15.78:0/441245529 wait complete. 2024-04-25T01:35:23.625 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.983+0000 7f44bb90c700 1 Processor -- start 2024-04-25T01:35:23.625 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.983+0000 7f44bb90c700 1 -- start start 2024-04-25T01:35:23.625 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.983+0000 7f44bb90c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4105fd0 0x7f44b419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:23.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.983+0000 7f44bb90c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4109490 con 0x7f44b4105fd0 2024-04-25T01:35:23.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.984+0000 7f44b96a8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4105fd0 0x7f44b419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:23.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.984+0000 7f44b96a8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4105fd0 0x7f44b419b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37584/0 (socket says 172.21.15.78:37584) 2024-04-25T01:35:23.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.984+0000 7f44b96a8700 1 -- 172.21.15.78:0/101060322 learned_addr learned my addr 172.21.15.78:0/101060322 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:23.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.984+0000 7f44b96a8700 1 -- 172.21.15.78:0/101060322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b419b910 con 0x7f44b4105fd0 2024-04-25T01:35:23.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.984+0000 7f44b96a8700 1 --2- 172.21.15.78:0/101060322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4105fd0 0x7f44b419b3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f44b0000c00 tx=0x7f44b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:23.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.984+0000 7f44aa7fc700 1 -- 172.21.15.78:0/101060322 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44b0003db0 con 0x7f44b4105fd0 2024-04-25T01:35:23.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.984+0000 7f44aa7fc700 1 -- 172.21.15.78:0/101060322 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44b0011630 con 0x7f44b4105fd0 2024-04-25T01:35:23.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.984+0000 7f44aa7fc700 1 -- 172.21.15.78:0/101060322 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44b0011950 con 0x7f44b4105fd0 2024-04-25T01:35:23.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.984+0000 7f44bb90c700 1 -- 172.21.15.78:0/101060322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b419bba0 con 0x7f44b4105fd0 2024-04-25T01:35:23.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.984+0000 7f44bb90c700 1 -- 172.21.15.78:0/101060322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b419e890 con 0x7f44b4105fd0 2024-04-25T01:35:23.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.984+0000 7f44aa7fc700 1 -- 172.21.15.78:0/101060322 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f44b0011af0 con 0x7f44b4105fd0 2024-04-25T01:35:23.626 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.985+0000 7f44aa7fc700 1 -- 172.21.15.78:0/101060322 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f44b001c450 con 0x7f44b4105fd0 2024-04-25T01:35:23.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.985+0000 7f44bb90c700 1 -- 172.21.15.78:0/101060322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b4063890 con 0x7f44b4105fd0 2024-04-25T01:35:23.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:22.988+0000 7f44aa7fc700 1 -- 172.21.15.78:0/101060322 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f44b0032020 con 0x7f44b4105fd0 2024-04-25T01:35:23.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:23.058+0000 7f44bb90c700 1 -- 172.21.15.78:0/101060322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f44b402dd40 con 0x7f44b4105fd0 2024-04-25T01:35:23.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:23.058+0000 7f44aa7fc700 1 -- 172.21.15.78:0/101060322 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f44b0004260 con 0x7f44b4105fd0 2024-04-25T01:35:23.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:23.059+0000 7f44bb90c700 1 -- 172.21.15.78:0/101060322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4105fd0 msgr2=0x7f44b419b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:23.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:23.059+0000 7f44bb90c700 1 --2- 172.21.15.78:0/101060322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4105fd0 0x7f44b419b3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f44b0000c00 tx=0x7f44b0004750 comp rx=0 tx=0).stop 2024-04-25T01:35:23.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:23.059+0000 7f44bb90c700 1 -- 172.21.15.78:0/101060322 shutdown_connections 2024-04-25T01:35:23.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:23.059+0000 7f44bb90c700 1 --2- 172.21.15.78:0/101060322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44b4105fd0 0x7f44b419b3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:23.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:23.059+0000 7f44bb90c700 1 -- 172.21.15.78:0/101060322 >> 172.21.15.78:0/101060322 conn(0x7f44b4101900 msgr2=0x7f44b4192060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:23.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:23.059+0000 7f44bb90c700 1 -- 172.21.15.78:0/101060322 shutdown_connections 2024-04-25T01:35:23.627 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:23.059+0000 7f44bb90c700 1 -- 172.21.15.78:0/101060322 wait complete. 2024-04-25T01:35:23.627 INFO:teuthology.orchestra.run.smithi078.stdout:mon is available 2024-04-25T01:35:23.628 INFO:teuthology.orchestra.run.smithi078.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T01:35:24.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T01:35:24.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T01:35:24.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout fsid = df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:35:24.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-04-25T01:35:24.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T01:35:24.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T01:35:24.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T01:35:24.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T01:35:24.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T01:35:24.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T01:35:24.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T01:35:24.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T01:35:24.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T01:35:24.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T01:35:24.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T01:35:24.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T01:35:24.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T01:35:24.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T01:35:24.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.122+0000 7fd64fad9700 1 Processor -- start 2024-04-25T01:35:24.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.122+0000 7fd64fad9700 1 -- start start 2024-04-25T01:35:24.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.122+0000 7fd64fad9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6481065a0 0x7fd6481069c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:24.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.123+0000 7fd64fad9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd64807c650 con 0x7fd6481065a0 2024-04-25T01:35:24.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.123+0000 7fd64d875700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6481065a0 0x7fd6481069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:24.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.123+0000 7fd64d875700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6481065a0 0x7fd6481069c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37594/0 (socket says 172.21.15.78:37594) 2024-04-25T01:35:24.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.123+0000 7fd64d875700 1 -- 172.21.15.78:0/405807900 learned_addr learned my addr 172.21.15.78:0/405807900 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:24.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.123+0000 7fd64d875700 1 -- 172.21.15.78:0/405807900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd648106f00 con 0x7fd6481065a0 2024-04-25T01:35:24.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.123+0000 7fd64d875700 1 --2- 172.21.15.78:0/405807900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6481065a0 0x7fd6481069c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd6380098a0 tx=0x7fd638009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae7ee4b55fdde5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:24.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.123+0000 7fd64c873700 1 -- 172.21.15.78:0/405807900 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd638004160 con 0x7fd6481065a0 2024-04-25T01:35:24.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.123+0000 7fd64c873700 1 -- 172.21.15.78:0/405807900 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd638004300 con 0x7fd6481065a0 2024-04-25T01:35:24.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.123+0000 7fd64c873700 1 -- 172.21.15.78:0/405807900 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6380036a0 con 0x7fd6481065a0 2024-04-25T01:35:24.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.124+0000 7fd64fad9700 1 -- 172.21.15.78:0/405807900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6481065a0 msgr2=0x7fd6481069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:24.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.124+0000 7fd64fad9700 1 --2- 172.21.15.78:0/405807900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6481065a0 0x7fd6481069c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd6380098a0 tx=0x7fd638009bb0 comp rx=0 tx=0).stop 2024-04-25T01:35:24.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.124+0000 7fd64fad9700 1 -- 172.21.15.78:0/405807900 shutdown_connections 2024-04-25T01:35:24.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.124+0000 7fd64fad9700 1 --2- 172.21.15.78:0/405807900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6481065a0 0x7fd6481069c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.124+0000 7fd64fad9700 1 -- 172.21.15.78:0/405807900 >> 172.21.15.78:0/405807900 conn(0x7fd6481021a0 msgr2=0x7fd6481045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.124+0000 7fd64fad9700 1 -- 172.21.15.78:0/405807900 shutdown_connections 2024-04-25T01:35:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.124+0000 7fd64fad9700 1 -- 172.21.15.78:0/405807900 wait complete. 2024-04-25T01:35:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.124+0000 7fd64fad9700 1 Processor -- start 2024-04-25T01:35:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.125+0000 7fd64fad9700 1 -- start start 2024-04-25T01:35:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.125+0000 7fd64fad9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6481065a0 0x7fd64819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.125+0000 7fd64fad9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd648107430 con 0x7fd6481065a0 2024-04-25T01:35:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.125+0000 7fd64d875700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6481065a0 0x7fd64819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.125+0000 7fd64d875700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6481065a0 0x7fd64819d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37596/0 (socket says 172.21.15.78:37596) 2024-04-25T01:35:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.125+0000 7fd64d875700 1 -- 172.21.15.78:0/967044778 learned_addr learned my addr 172.21.15.78:0/967044778 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.125+0000 7fd64d875700 1 -- 172.21.15.78:0/967044778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64819dae0 con 0x7fd6481065a0 2024-04-25T01:35:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.125+0000 7fd64d875700 1 --2- 172.21.15.78:0/967044778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6481065a0 0x7fd64819d5a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd638000c00 tx=0x7fd638004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:24.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.125+0000 7fd63effd700 1 -- 172.21.15.78:0/967044778 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd638003db0 con 0x7fd6481065a0 2024-04-25T01:35:24.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.125+0000 7fd63effd700 1 -- 172.21.15.78:0/967044778 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd638011630 con 0x7fd6481065a0 2024-04-25T01:35:24.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.125+0000 7fd63effd700 1 -- 172.21.15.78:0/967044778 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd638011950 con 0x7fd6481065a0 2024-04-25T01:35:24.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.125+0000 7fd64fad9700 1 -- 172.21.15.78:0/967044778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64819dd70 con 0x7fd6481065a0 2024-04-25T01:35:24.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.125+0000 7fd64fad9700 1 -- 172.21.15.78:0/967044778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6481a0a10 con 0x7fd6481065a0 2024-04-25T01:35:24.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.126+0000 7fd63effd700 1 -- 172.21.15.78:0/967044778 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd638011af0 con 0x7fd6481065a0 2024-04-25T01:35:24.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.126+0000 7fd63effd700 1 -- 172.21.15.78:0/967044778 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd63801c450 con 0x7fd6481065a0 2024-04-25T01:35:24.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.126+0000 7fd64fad9700 1 -- 172.21.15.78:0/967044778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd648063890 con 0x7fd6481065a0 2024-04-25T01:35:24.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.128+0000 7fd63effd700 1 -- 172.21.15.78:0/967044778 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd638032020 con 0x7fd6481065a0 2024-04-25T01:35:24.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.196+0000 7fd64fad9700 1 -- 172.21.15.78:0/967044778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd6481a0be0 con 0x7fd6481065a0 2024-04-25T01:35:24.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.199+0000 7fd63effd700 1 -- 172.21.15.78:0/967044778 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fd638004260 con 0x7fd6481065a0 2024-04-25T01:35:24.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.200+0000 7fd63effd700 1 -- 172.21.15.78:0/967044778 <== mon.0 v2:172.21.15.78:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd63801b450 con 0x7fd6481065a0 2024-04-25T01:35:24.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.201+0000 7fd64fad9700 1 -- 172.21.15.78:0/967044778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6481065a0 msgr2=0x7fd64819d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:24.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.201+0000 7fd64fad9700 1 --2- 172.21.15.78:0/967044778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6481065a0 0x7fd64819d5a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd638000c00 tx=0x7fd638004750 comp rx=0 tx=0).stop 2024-04-25T01:35:24.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.201+0000 7fd64fad9700 1 -- 172.21.15.78:0/967044778 shutdown_connections 2024-04-25T01:35:24.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.201+0000 7fd64fad9700 1 --2- 172.21.15.78:0/967044778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6481065a0 0x7fd64819d5a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:24.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.201+0000 7fd64fad9700 1 -- 172.21.15.78:0/967044778 >> 172.21.15.78:0/967044778 conn(0x7fd6481021a0 msgr2=0x7fd648193bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:24.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.201+0000 7fd64fad9700 1 -- 172.21.15.78:0/967044778 shutdown_connections 2024-04-25T01:35:24.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:24.201+0000 7fd64fad9700 1 -- 172.21.15.78:0/967044778 wait complete. 2024-04-25T01:35:24.668 INFO:teuthology.orchestra.run.smithi078.stdout:Generating new minimal ceph.conf... 2024-04-25T01:35:24.741 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:24 smithi078 ceph-mon[29089]: from='client.? 172.21.15.78:0/967044778' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T01:35:24.741 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:24 smithi078 ceph-mon[29089]: from='client.? 172.21.15.78:0/967044778' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T01:35:25.768 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:25 smithi078 ceph-mon[29089]: from='client.? 172.21.15.78:0/2137559349' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:35:25.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.119+0000 7fb6be5ca700 1 Processor -- start 2024-04-25T01:35:25.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.119+0000 7fb6be5ca700 1 -- start start 2024-04-25T01:35:25.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.119+0000 7fb6be5ca700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b8107da0 0x7fb6b81081c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:25.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.119+0000 7fb6be5ca700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b8108790 con 0x7fb6b8107da0 2024-04-25T01:35:25.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.119+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b8107da0 0x7fb6b81081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:25.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.119+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b8107da0 0x7fb6b81081c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37602/0 (socket says 172.21.15.78:37602) 2024-04-25T01:35:25.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.119+0000 7fb6b7fff700 1 -- 172.21.15.78:0/669292092 learned_addr learned my addr 172.21.15.78:0/669292092 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:25.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.119+0000 7fb6b7fff700 1 -- 172.21.15.78:0/669292092 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b8108f00 con 0x7fb6b8107da0 2024-04-25T01:35:25.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.120+0000 7fb6b7fff700 1 --2- 172.21.15.78:0/669292092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b8107da0 0x7fb6b81081c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80098a0 tx=0x7fb6a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10df32b71778e29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:25.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.120+0000 7fb6b6ffd700 1 -- 172.21.15.78:0/669292092 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a8004160 con 0x7fb6b8107da0 2024-04-25T01:35:25.775 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.120+0000 7fb6b6ffd700 1 -- 172.21.15.78:0/669292092 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb6a8004300 con 0x7fb6b8107da0 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.120+0000 7fb6b6ffd700 1 -- 172.21.15.78:0/669292092 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a8011b50 con 0x7fb6b8107da0 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.120+0000 7fb6be5ca700 1 -- 172.21.15.78:0/669292092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b8107da0 msgr2=0x7fb6b81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.120+0000 7fb6be5ca700 1 --2- 172.21.15.78:0/669292092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b8107da0 0x7fb6b81081c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80098a0 tx=0x7fb6a8009bb0 comp rx=0 tx=0).stop 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.120+0000 7fb6be5ca700 1 -- 172.21.15.78:0/669292092 shutdown_connections 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.120+0000 7fb6be5ca700 1 --2- 172.21.15.78:0/669292092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b8107da0 0x7fb6b81081c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.120+0000 7fb6be5ca700 1 -- 172.21.15.78:0/669292092 >> 172.21.15.78:0/669292092 conn(0x7fb6b8103360 msgr2=0x7fb6b8105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.120+0000 7fb6be5ca700 1 -- 172.21.15.78:0/669292092 shutdown_connections 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.120+0000 7fb6be5ca700 1 -- 172.21.15.78:0/669292092 wait complete. 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.121+0000 7fb6be5ca700 1 Processor -- start 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.121+0000 7fb6be5ca700 1 -- start start 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.121+0000 7fb6be5ca700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b8107da0 0x7fb6b819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.121+0000 7fb6be5ca700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b810c2b0 con 0x7fb6b8107da0 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.121+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b8107da0 0x7fb6b819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.121+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b8107da0 0x7fb6b819bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37614/0 (socket says 172.21.15.78:37614) 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.121+0000 7fb6b7fff700 1 -- 172.21.15.78:0/2137559349 learned_addr learned my addr 172.21.15.78:0/2137559349 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:25.776 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.122+0000 7fb6b7fff700 1 -- 172.21.15.78:0/2137559349 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b819c230 con 0x7fb6b8107da0 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.122+0000 7fb6b7fff700 1 --2- 172.21.15.78:0/2137559349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b8107da0 0x7fb6b819bcf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8011670 tx=0x7fb6a8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.122+0000 7fb6b57fa700 1 -- 172.21.15.78:0/2137559349 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a80039c0 con 0x7fb6b8107da0 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.122+0000 7fb6b57fa700 1 -- 172.21.15.78:0/2137559349 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb6a8011e50 con 0x7fb6b8107da0 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.122+0000 7fb6b57fa700 1 -- 172.21.15.78:0/2137559349 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a801a5c0 con 0x7fb6b8107da0 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.122+0000 7fb6be5ca700 1 -- 172.21.15.78:0/2137559349 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b819c4c0 con 0x7fb6b8107da0 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.122+0000 7fb6be5ca700 1 -- 172.21.15.78:0/2137559349 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b819f1a0 con 0x7fb6b8107da0 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.122+0000 7fb6b57fa700 1 -- 172.21.15.78:0/2137559349 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb6a801ac50 con 0x7fb6b8107da0 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.122+0000 7fb6b57fa700 1 -- 172.21.15.78:0/2137559349 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb6a8019700 con 0x7fb6b8107da0 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.123+0000 7fb6be5ca700 1 -- 172.21.15.78:0/2137559349 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b8063890 con 0x7fb6b8107da0 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.125+0000 7fb6b57fa700 1 -- 172.21.15.78:0/2137559349 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb6a8003b60 con 0x7fb6b8107da0 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.192+0000 7fb6be5ca700 1 -- 172.21.15.78:0/2137559349 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb6b819f370 con 0x7fb6b8107da0 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.192+0000 7fb6b57fa700 1 -- 172.21.15.78:0/2137559349 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fb6a8019950 con 0x7fb6b8107da0 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.193+0000 7fb6be5ca700 1 -- 172.21.15.78:0/2137559349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b8107da0 msgr2=0x7fb6b819bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.193+0000 7fb6be5ca700 1 --2- 172.21.15.78:0/2137559349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b8107da0 0x7fb6b819bcf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8011670 tx=0x7fb6a8011b60 comp rx=0 tx=0).stop 2024-04-25T01:35:25.777 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.193+0000 7fb6be5ca700 1 -- 172.21.15.78:0/2137559349 shutdown_connections 2024-04-25T01:35:25.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.193+0000 7fb6be5ca700 1 --2- 172.21.15.78:0/2137559349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6b8107da0 0x7fb6b819bcf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:25.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.193+0000 7fb6be5ca700 1 -- 172.21.15.78:0/2137559349 >> 172.21.15.78:0/2137559349 conn(0x7fb6b8103360 msgr2=0x7fb6b8103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:25.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.193+0000 7fb6be5ca700 1 -- 172.21.15.78:0/2137559349 shutdown_connections 2024-04-25T01:35:25.778 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:25.193+0000 7fb6be5ca700 1 -- 172.21.15.78:0/2137559349 wait complete. 2024-04-25T01:35:25.778 INFO:teuthology.orchestra.run.smithi078.stdout:Restarting the monitor... 2024-04-25T01:35:26.038 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:25 smithi078 systemd[1]: Stopping Ceph mon.a for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:35:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:26 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a[29078]: 2024-04-25T01:35:26.061+0000 7f54db31e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:35:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:26 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a[29078]: 2024-04-25T01:35:26.061+0000 7f54db31e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T01:35:26.422 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:26 smithi078 podman[29339]: 2024-04-25 01:35:26.226585599 +0000 UTC m=+0.261967405 container died 4fe595d235eb62c998b3eba8a175ecb311227709994cb201a3726497742f2379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True) 2024-04-25T01:35:26.696 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:26 smithi078 podman[29339]: 2024-04-25 01:35:26.545466231 +0000 UTC m=+0.580848043 container cleanup 4fe595d235eb62c998b3eba8a175ecb311227709994cb201a3726497742f2379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T01:35:26.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:26 smithi078 bash[29339]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a 2024-04-25T01:35:26.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:26 smithi078 podman[29353]: 2024-04-25 01:35:26.72135692 +0000 UTC m=+0.495137757 container remove 4fe595d235eb62c998b3eba8a175ecb311227709994cb201a3726497742f2379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:35:26.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:26 smithi078 systemd[1]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T01:35:26.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:26 smithi078 systemd[1]: Stopped Ceph mon.a for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:35:26.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:26 smithi078 systemd[1]: Starting Ceph mon.a for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:35:27.275 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 podman[29450]: 2024-04-25 01:35:27.047352291 +0000 UTC m=+0.100548205 container create c83da7f37b2e468370b9b4712e1debf357ad9c2fb4b2bb3eeea92ab0628e9004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T01:35:27.275 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 podman[29450]: 2024-04-25 01:35:26.96506172 +0000 UTC m=+0.018257645 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:35:27.275 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 podman[29450]: 2024-04-25 01:35:27.274461065 +0000 UTC m=+0.327656978 container init c83da7f37b2e468370b9b4712e1debf357ad9c2fb4b2bb3eeea92ab0628e9004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:35:27.276 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 podman[29450]: 2024-04-25 01:35:27.277280202 +0000 UTC m=+0.330476115 container start c83da7f37b2e468370b9b4712e1debf357ad9c2fb4b2bb3eeea92ab0628e9004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T01:35:27.276 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T01:35:27.276 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-25T01:35:27.338 INFO:teuthology.orchestra.run.smithi078.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T01:35:27.544 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: pidfile_write: ignore empty --pid-file 2024-04-25T01:35:27.544 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: load: jerasure load: lrc 2024-04-25T01:35:27.544 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: RocksDB version: 7.9.2 2024-04-25T01:35:27.544 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Git sha 0 2024-04-25T01:35:27.544 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-25T01:35:27.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: DB SUMMARY 2024-04-25T01:35:27.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: DB Session ID: XYFOWOI3RXWV6TJU3OXL 2024-04-25T01:35:27.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: CURRENT file: CURRENT 2024-04-25T01:35:27.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T01:35:27.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T01:35:27.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T01:35:27.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105416 ; 2024-04-25T01:35:27.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.error_if_exists: 0 2024-04-25T01:35:27.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.create_if_missing: 0 2024-04-25T01:35:27.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.paranoid_checks: 1 2024-04-25T01:35:27.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T01:35:27.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T01:35:27.545 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T01:35:27.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.env: 0x55df59400c60 2024-04-25T01:35:27.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T01:35:27.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.info_log: 0x55df5a777aa0 2024-04-25T01:35:27.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T01:35:27.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.statistics: (nil) 2024-04-25T01:35:27.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.use_fsync: 0 2024-04-25T01:35:27.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_log_file_size: 0 2024-04-25T01:35:27.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T01:35:27.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T01:35:27.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T01:35:27.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T01:35:27.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.allow_fallocate: 1 2024-04-25T01:35:27.546 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T01:35:27.547 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T01:35:27.547 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.use_direct_reads: 0 2024-04-25T01:35:27.547 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T01:35:27.547 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T01:35:27.547 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.db_log_dir: 2024-04-25T01:35:27.547 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.wal_dir: 2024-04-25T01:35:27.547 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T01:35:27.547 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T01:35:27.547 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T01:35:27.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T01:35:27.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T01:35:27.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T01:35:27.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T01:35:27.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T01:35:27.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.write_buffer_manager: 0x55df5a74a1e0 2024-04-25T01:35:27.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T01:35:27.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T01:35:27.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T01:35:27.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T01:35:27.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T01:35:27.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T01:35:27.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T01:35:27.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T01:35:27.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.unordered_write: 0 2024-04-25T01:35:27.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T01:35:27.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T01:35:27.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T01:35:27.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T01:35:27.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.row_cache: None 2024-04-25T01:35:27.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.wal_filter: None 2024-04-25T01:35:27.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T01:35:27.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T01:35:27.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.two_write_queues: 0 2024-04-25T01:35:27.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T01:35:27.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.wal_compression: 0 2024-04-25T01:35:27.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.atomic_flush: 0 2024-04-25T01:35:27.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T01:35:27.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T01:35:27.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T01:35:27.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.log_readahead_size: 0 2024-04-25T01:35:27.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T01:35:27.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T01:35:27.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T01:35:27.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T01:35:27.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T01:35:27.551 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T01:35:27.552 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T01:35:27.552 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_background_jobs: 2 2024-04-25T01:35:27.552 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_background_compactions: -1 2024-04-25T01:35:27.552 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_subcompactions: 1 2024-04-25T01:35:27.552 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T01:35:27.552 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T01:35:27.552 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T01:35:27.552 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T01:35:27.552 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T01:35:27.552 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T01:35:27.552 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T01:35:27.552 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T01:35:27.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_open_files: -1 2024-04-25T01:35:27.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T01:35:27.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T01:35:27.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T01:35:27.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T01:35:27.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_background_flushes: -1 2024-04-25T01:35:27.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Compression algorithms supported: 2024-04-25T01:35:27.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T01:35:27.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: kZSTD supported: 0 2024-04-25T01:35:27.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: kXpressCompression supported: 0 2024-04-25T01:35:27.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T01:35:27.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: kZlibCompression supported: 1 2024-04-25T01:35:27.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: kSnappyCompression supported: 1 2024-04-25T01:35:27.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: kLZ4Compression supported: 1 2024-04-25T01:35:27.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: kBZip2Compression supported: 0 2024-04-25T01:35:27.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T01:35:27.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T01:35:27.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T01:35:27.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T01:35:27.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T01:35:27.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.merge_operator: 2024-04-25T01:35:27.554 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compaction_filter: None 2024-04-25T01:35:27.555 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T01:35:27.555 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T01:35:27.555 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T01:35:27.555 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T01:35:27.555 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55df5a777ba0) 2024-04-25T01:35:27.555 INFO:journalctl@ceph.mon.a.smithi078.stdout: cache_index_and_filter_blocks: 1 2024-04-25T01:35:27.555 INFO:journalctl@ceph.mon.a.smithi078.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T01:35:27.555 INFO:journalctl@ceph.mon.a.smithi078.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T01:35:27.555 INFO:journalctl@ceph.mon.a.smithi078.stdout: pin_top_level_index_and_filter: 1 2024-04-25T01:35:27.555 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_type: 0 2024-04-25T01:35:27.555 INFO:journalctl@ceph.mon.a.smithi078.stdout: data_block_index_type: 0 2024-04-25T01:35:27.555 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_shortening: 1 2024-04-25T01:35:27.555 INFO:journalctl@ceph.mon.a.smithi078.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T01:35:27.555 INFO:journalctl@ceph.mon.a.smithi078.stdout: checksum: 4 2024-04-25T01:35:27.556 INFO:journalctl@ceph.mon.a.smithi078.stdout: no_block_cache: 0 2024-04-25T01:35:27.556 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache: 0x55df5a77b090 2024-04-25T01:35:27.556 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_name: BinnedLRUCache 2024-04-25T01:35:27.556 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_options: 2024-04-25T01:35:27.556 INFO:journalctl@ceph.mon.a.smithi078.stdout: capacity : 536870912 2024-04-25T01:35:27.556 INFO:journalctl@ceph.mon.a.smithi078.stdout: num_shard_bits : 4 2024-04-25T01:35:27.556 INFO:journalctl@ceph.mon.a.smithi078.stdout: strict_capacity_limit : 0 2024-04-25T01:35:27.556 INFO:journalctl@ceph.mon.a.smithi078.stdout: high_pri_pool_ratio: 0.000 2024-04-25T01:35:27.556 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_compressed: (nil) 2024-04-25T01:35:27.556 INFO:journalctl@ceph.mon.a.smithi078.stdout: persistent_cache: (nil) 2024-04-25T01:35:27.556 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_size: 4096 2024-04-25T01:35:27.556 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_size_deviation: 10 2024-04-25T01:35:27.556 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_restart_interval: 16 2024-04-25T01:35:27.556 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_block_restart_interval: 1 2024-04-25T01:35:27.557 INFO:journalctl@ceph.mon.a.smithi078.stdout: metadata_block_size: 4096 2024-04-25T01:35:27.557 INFO:journalctl@ceph.mon.a.smithi078.stdout: partition_filters: 0 2024-04-25T01:35:27.557 INFO:journalctl@ceph.mon.a.smithi078.stdout: use_delta_encoding: 1 2024-04-25T01:35:27.557 INFO:journalctl@ceph.mon.a.smithi078.stdout: filter_policy: bloomfilter 2024-04-25T01:35:27.557 INFO:journalctl@ceph.mon.a.smithi078.stdout: whole_key_filtering: 1 2024-04-25T01:35:27.557 INFO:journalctl@ceph.mon.a.smithi078.stdout: verify_compression: 0 2024-04-25T01:35:27.557 INFO:journalctl@ceph.mon.a.smithi078.stdout: read_amp_bytes_per_bit: 0 2024-04-25T01:35:27.557 INFO:journalctl@ceph.mon.a.smithi078.stdout: format_version: 5 2024-04-25T01:35:27.557 INFO:journalctl@ceph.mon.a.smithi078.stdout: enable_index_compression: 1 2024-04-25T01:35:27.557 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_align: 0 2024-04-25T01:35:27.557 INFO:journalctl@ceph.mon.a.smithi078.stdout: max_auto_readahead_size: 262144 2024-04-25T01:35:27.557 INFO:journalctl@ceph.mon.a.smithi078.stdout: prepopulate_block_cache: 0 2024-04-25T01:35:27.557 INFO:journalctl@ceph.mon.a.smithi078.stdout: initial_auto_readahead_size: 8192 2024-04-25T01:35:27.558 INFO:journalctl@ceph.mon.a.smithi078.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T01:35:27.558 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T01:35:27.558 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T01:35:27.558 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compression: NoCompression 2024-04-25T01:35:27.558 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T01:35:27.558 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T01:35:27.558 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T01:35:27.558 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.num_levels: 7 2024-04-25T01:35:27.558 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T01:35:27.558 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T01:35:27.559 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T01:35:27.559 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T01:35:27.559 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T01:35:27.559 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T01:35:27.559 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T01:35:27.559 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T01:35:27.559 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T01:35:27.559 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T01:35:27.559 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T01:35:27.559 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T01:35:27.559 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T01:35:27.559 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T01:35:27.559 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T01:35:27.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T01:35:27.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T01:35:27.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T01:35:27.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T01:35:27.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T01:35:27.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T01:35:27.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T01:35:27.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T01:35:27.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T01:35:27.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T01:35:27.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T01:35:27.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T01:35:27.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T01:35:27.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T01:35:27.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T01:35:27.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T01:35:27.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T01:35:27.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T01:35:27.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T01:35:27.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T01:35:27.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T01:35:27.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T01:35:27.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T01:35:27.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T01:35:27.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T01:35:27.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T01:35:27.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T01:35:27.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T01:35:27.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T01:35:27.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T01:35:27.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T01:35:27.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T01:35:27.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T01:35:27.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T01:35:27.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T01:35:27.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T01:35:27.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T01:35:27.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T01:35:27.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.table_properties_collectors: 2024-04-25T01:35:27.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.inplace_update_support: 0 2024-04-25T01:35:27.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T01:35:27.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T01:35:27.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T01:35:27.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T01:35:27.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.bloom_locality: 0 2024-04-25T01:35:27.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.max_successive_merges: 0 2024-04-25T01:35:27.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T01:35:27.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T01:35:27.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T01:35:27.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T01:35:27.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.ttl: 2592000 2024-04-25T01:35:27.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T01:35:27.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T01:35:27.563 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T01:35:27.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.enable_blob_files: false 2024-04-25T01:35:27.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.min_blob_size: 0 2024-04-25T01:35:27.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T01:35:27.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T01:35:27.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T01:35:27.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T01:35:27.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T01:35:27.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T01:35:27.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T01:35:27.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T01:35:27.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T01:35:27.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T01:35:27.564 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a9122b82-85b9-4556-afe1-69d48f451d84 2024-04-25T01:35:27.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714008927305926, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T01:35:27.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T01:35:27.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714008927306929, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100481, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98392, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14328, "raw_average_key_size": 49, "raw_value_size": 90603, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714008927, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a9122b82-85b9-4556-afe1-69d48f451d84", "db_session_id": "XYFOWOI3RXWV6TJU3OXL", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T01:35:27.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714008927307009, "job": 1, "event": "recovery_finished"} 2024-04-25T01:35:27.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T01:35:27.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T01:35:27.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55df5a86c000 2024-04-25T01:35:27.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: DB pointer 0x55df5a856000 2024-04-25T01:35:27.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T01:35:27.565 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T01:35:27.565 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** DB Stats ** 2024-04-25T01:35:27.565 INFO:journalctl@ceph.mon.a.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T01:35:27.565 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T01:35:27.566 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T01:35:27.566 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T01:35:27.566 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T01:35:27.566 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T01:35:27.566 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T01:35:27.566 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-04-25T01:35:27.566 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** Compaction Stats [default] ** 2024-04-25T01:35:27.566 INFO:journalctl@ceph.mon.a.smithi078.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T01:35:27.566 INFO:journalctl@ceph.mon.a.smithi078.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T01:35:27.566 INFO:journalctl@ceph.mon.a.smithi078.stdout: L0 2/0 99.96 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T01:35:27.566 INFO:journalctl@ceph.mon.a.smithi078.stdout: Sum 2/0 99.96 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T01:35:27.566 INFO:journalctl@ceph.mon.a.smithi078.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T01:35:27.566 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-04-25T01:35:27.566 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** Compaction Stats [default] ** 2024-04-25T01:35:27.567 INFO:journalctl@ceph.mon.a.smithi078.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T01:35:27.567 INFO:journalctl@ceph.mon.a.smithi078.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T01:35:27.567 INFO:journalctl@ceph.mon.a.smithi078.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 150.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T01:35:27.567 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-04-25T01:35:27.567 INFO:journalctl@ceph.mon.a.smithi078.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T01:35:27.567 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-04-25T01:35:27.567 INFO:journalctl@ceph.mon.a.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T01:35:27.567 INFO:journalctl@ceph.mon.a.smithi078.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T01:35:27.567 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T01:35:27.567 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T01:35:27.567 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T01:35:27.567 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T01:35:27.567 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative compaction: 0.00 GB write, 32.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T01:35:27.568 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval compaction: 0.00 GB write, 32.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T01:35:27.568 INFO:journalctl@ceph.mon.a.smithi078.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T01:35:27.568 INFO:journalctl@ceph.mon.a.smithi078.stdout: Block cache BinnedLRUCache@0x55df5a77b090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T01:35:27.568 INFO:journalctl@ceph.mon.a.smithi078.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T01:35:27.568 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-04-25T01:35:27.568 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T01:35:27.568 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: starting mon.a rank 0 at public addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] at bind addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:35:27.568 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: mon.a@-1(???) e1 preinit fsid df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:35:27.568 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: mon.a@-1(???).mds e1 new map 2024-04-25T01:35:27.568 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: mon.a@-1(???).mds e1 print_map 2024-04-25T01:35:27.568 INFO:journalctl@ceph.mon.a.smithi078.stdout: e1 2024-04-25T01:35:27.568 INFO:journalctl@ceph.mon.a.smithi078.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:35:27.568 INFO:journalctl@ceph.mon.a.smithi078.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T01:35:27.569 INFO:journalctl@ceph.mon.a.smithi078.stdout: legacy client fscid: -1 2024-04-25T01:35:27.569 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-04-25T01:35:27.569 INFO:journalctl@ceph.mon.a.smithi078.stdout: No filesystems configured 2024-04-25T01:35:27.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T01:35:27.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T01:35:27.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T01:35:27.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T01:35:27.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T01:35:27.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 bash[29450]: c83da7f37b2e468370b9b4712e1debf357ad9c2fb4b2bb3eeea92ab0628e9004 2024-04-25T01:35:27.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 systemd[1]: Started Ceph mon.a for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:35:27.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T01:35:27.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: monmap epoch 1 2024-04-25T01:35:27.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: fsid df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:35:27.569 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: last_changed 2024-04-25T01:35:19.558079+0000 2024-04-25T01:35:27.570 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: created 2024-04-25T01:35:19.558079+0000 2024-04-25T01:35:27.570 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: min_mon_release 19 (squid) 2024-04-25T01:35:27.570 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: election_strategy: 1 2024-04-25T01:35:27.570 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T01:35:27.570 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: fsmap 2024-04-25T01:35:27.570 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T01:35:27.570 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:27 smithi078 ceph-mon[29469]: mgrmap e1: no daemons active 2024-04-25T01:35:28.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.761+0000 7f8798670700 1 Processor -- start 2024-04-25T01:35:28.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.761+0000 7f8798670700 1 -- start start 2024-04-25T01:35:28.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.761+0000 7f8798670700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f879007e670 0x7f879007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.761+0000 7f8798670700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f879007d0b0 con 0x7f879007e670 2024-04-25T01:35:28.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.761+0000 7f879640c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f879007e670 0x7f879007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.382 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.761+0000 7f879640c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f879007e670 0x7f879007cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37616/0 (socket says 172.21.15.78:37616) 2024-04-25T01:35:28.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.761+0000 7f879640c700 1 -- 172.21.15.78:0/2164588289 learned_addr learned my addr 172.21.15.78:0/2164588289 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:28.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.762+0000 7f879640c700 1 -- 172.21.15.78:0/2164588289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879007d230 con 0x7f879007e670 2024-04-25T01:35:28.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.762+0000 7f879640c700 1 --2- 172.21.15.78:0/2164588289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f879007e670 0x7f879007cb70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f878c0098a0 tx=0x7f878c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec77857703dc3e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.762+0000 7f879540a700 1 -- 172.21.15.78:0/2164588289 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f878c004160 con 0x7f879007e670 2024-04-25T01:35:28.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.762+0000 7f879540a700 1 -- 172.21.15.78:0/2164588289 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f878c004300 con 0x7f879007e670 2024-04-25T01:35:28.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.762+0000 7f879540a700 1 -- 172.21.15.78:0/2164588289 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f878c011b50 con 0x7f879007e670 2024-04-25T01:35:28.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.763+0000 7f8798670700 1 -- 172.21.15.78:0/2164588289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f879007e670 msgr2=0x7f879007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.763+0000 7f8798670700 1 --2- 172.21.15.78:0/2164588289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f879007e670 0x7f879007cb70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f878c0098a0 tx=0x7f878c009bb0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.763+0000 7f8798670700 1 -- 172.21.15.78:0/2164588289 shutdown_connections 2024-04-25T01:35:28.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.763+0000 7f8798670700 1 --2- 172.21.15.78:0/2164588289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f879007e670 0x7f879007cb70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.763+0000 7f8798670700 1 -- 172.21.15.78:0/2164588289 >> 172.21.15.78:0/2164588289 conn(0x7f8790103320 msgr2=0x7f8790105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:28.383 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.763+0000 7f8798670700 1 -- 172.21.15.78:0/2164588289 shutdown_connections 2024-04-25T01:35:28.384 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.763+0000 7f8798670700 1 -- 172.21.15.78:0/2164588289 wait complete. 2024-04-25T01:35:28.384 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.763+0000 7f8798670700 1 Processor -- start 2024-04-25T01:35:28.384 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.763+0000 7f8798670700 1 -- start start 2024-04-25T01:35:28.384 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.763+0000 7f8798670700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f879007e670 0x7f879019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:28.384 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.763+0000 7f8798670700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f879010e310 con 0x7f879007e670 2024-04-25T01:35:28.384 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.764+0000 7f879640c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f879007e670 0x7f879019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:28.384 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.764+0000 7f879640c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f879007e670 0x7f879019de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37632/0 (socket says 172.21.15.78:37632) 2024-04-25T01:35:28.384 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.764+0000 7f879640c700 1 -- 172.21.15.78:0/419534275 learned_addr learned my addr 172.21.15.78:0/419534275 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:28.384 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.764+0000 7f879640c700 1 -- 172.21.15.78:0/419534275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879019e3a0 con 0x7f879007e670 2024-04-25T01:35:28.384 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.764+0000 7f879640c700 1 --2- 172.21.15.78:0/419534275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f879007e670 0x7f879019de60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f878c011b60 tx=0x7f878c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:28.384 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.764+0000 7f87877fe700 1 -- 172.21.15.78:0/419534275 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f878c003b20 con 0x7f879007e670 2024-04-25T01:35:28.384 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.764+0000 7f87877fe700 1 -- 172.21.15.78:0/419534275 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f878c003cc0 con 0x7f879007e670 2024-04-25T01:35:28.384 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.764+0000 7f87877fe700 1 -- 172.21.15.78:0/419534275 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f878c024840 con 0x7f879007e670 2024-04-25T01:35:28.385 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.764+0000 7f8798670700 1 -- 172.21.15.78:0/419534275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879019e630 con 0x7f879007e670 2024-04-25T01:35:28.385 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.764+0000 7f8798670700 1 -- 172.21.15.78:0/419534275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87901a1320 con 0x7f879007e670 2024-04-25T01:35:28.385 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.764+0000 7f87877fe700 1 -- 172.21.15.78:0/419534275 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f878c022020 con 0x7f879007e670 2024-04-25T01:35:28.385 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.764+0000 7f87877fe700 1 -- 172.21.15.78:0/419534275 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f878c011e00 con 0x7f879007e670 2024-04-25T01:35:28.385 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.765+0000 7f8798670700 1 -- 172.21.15.78:0/419534275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8790063890 con 0x7f879007e670 2024-04-25T01:35:28.385 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.767+0000 7f87877fe700 1 -- 172.21.15.78:0/419534275 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f878c043dd0 con 0x7f879007e670 2024-04-25T01:35:28.385 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.834+0000 7f8798670700 1 -- 172.21.15.78:0/419534275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f87901a1a00 con 0x7f879007e670 2024-04-25T01:35:28.385 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.838+0000 7f87877fe700 1 -- 172.21.15.78:0/419534275 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f878c032020 con 0x7f879007e670 2024-04-25T01:35:28.385 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.839+0000 7f8798670700 1 -- 172.21.15.78:0/419534275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f879007e670 msgr2=0x7f879019de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:28.385 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.839+0000 7f8798670700 1 --2- 172.21.15.78:0/419534275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f879007e670 0x7f879019de60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f878c011b60 tx=0x7f878c011c40 comp rx=0 tx=0).stop 2024-04-25T01:35:28.385 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.839+0000 7f8798670700 1 -- 172.21.15.78:0/419534275 shutdown_connections 2024-04-25T01:35:28.385 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.839+0000 7f8798670700 1 --2- 172.21.15.78:0/419534275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f879007e670 0x7f879019de60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:28.385 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.839+0000 7f8798670700 1 -- 172.21.15.78:0/419534275 >> 172.21.15.78:0/419534275 conn(0x7f8790103320 msgr2=0x7f8790103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:28.386 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.839+0000 7f8798670700 1 -- 172.21.15.78:0/419534275 shutdown_connections 2024-04-25T01:35:28.386 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:27.839+0000 7f8798670700 1 -- 172.21.15.78:0/419534275 wait complete. 2024-04-25T01:35:28.386 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T01:35:28.408 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:35:28.408 INFO:teuthology.orchestra.run.smithi078.stdout:Creating mgr... 2024-04-25T01:35:28.409 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T01:35:28.409 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T01:35:28.650 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.x 2024-04-25T01:35:28.650 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to reset failed state of unit ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T01:35:28.794 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-df36bea8-02a3-11ef-bc93-c7b262605968.target.wants/ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-df36bea8-02a3-11ef-bc93-c7b262605968@.service. 2024-04-25T01:35:28.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:28 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/419534275' entity='client.admin' 2024-04-25T01:35:29.314 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:29 smithi078 podman[29684]: 2024-04-25 01:35:29.257287025 +0000 UTC m=+0.319172096 container start 20678b9d90e14170a3ec1565ac720f9405a418229ef573ad6aaaacea445d0720 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T01:35:29.314 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:29 smithi078 bash[29684]: 20678b9d90e14170a3ec1565ac720f9405a418229ef573ad6aaaacea445d0720 2024-04-25T01:35:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T01:35:29.327 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-04-25T01:35:29.334 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T01:35:29.334 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-04-25T01:35:29.334 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-04-25T01:35:29.334 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T01:35:29.342 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T01:35:29.342 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-04-25T01:35:29.350 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T01:35:29.350 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-04-25T01:35:29.350 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-04-25T01:35:29.350 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T01:35:29.350 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr to start... 2024-04-25T01:35:29.350 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr... 2024-04-25T01:35:29.570 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:29 smithi078 systemd[1]: Started Ceph mgr.x for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:35:29.571 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:29 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:29.409+0000 7f22ba3b6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:35:29.571 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:29 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:29.594+0000 7f22ba3b6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:35:29.880 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:29 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:29.700+0000 7f22ba3b6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:35:30.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:29 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1034685946' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T01:35:30.241 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:30 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:30.027+0000 7f22ba3b6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:35:30.242 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:30 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:30.113+0000 7f22ba3b6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:35:31.729 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:31 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:31.361+0000 7f22ba3b6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:35:31.729 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:31 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:31.533+0000 7f22ba3b6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:35:31.729 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:31 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:31.703+0000 7f22ba3b6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:35:31.991 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:31 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:31.991+0000 7f22ba3b6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:35:32.269 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T01:35:32.269 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:35:32.270 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "df36bea8-02a3-11ef-bc93-c7b262605968", 2024-04-25T01:35:32.270 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T01:35:32.270 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T01:35:32.270 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T01:35:32.270 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T01:35:32.270 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:35:32.270 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T01:35:32.270 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T01:35:32.270 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-04-25T01:35:32.270 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:35:32.270 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T01:35:32.270 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T01:35:32.270 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:35:32.271 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T01:35:32.271 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T01:35:32.271 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:35:32.271 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T01:35:32.271 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T01:35:32.271 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:35:32.271 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T01:35:32.271 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:35:32.271 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T01:35:32.271 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T01:35:32.271 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T01:35:32.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T01:35:32.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T01:35:32.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T01:35:32.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:35:32.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T01:35:32.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T01:35:32.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T01:35:32.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T01:35:32.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T01:35:32.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T01:35:32.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T01:35:32.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T01:35:32.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T01:35:32.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:35:32.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T01:35:32.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:35:32.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T01:35:32.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T01:35:32.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:35:32.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T01:35:32.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T01:35:32.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T01:35:32.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T01:35:32.281 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T01:35:32.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T01:35:32.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T01:35:32.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:35:32.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:35:32.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:35:32.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T01:35:32.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:35:32.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T01:35:22.529621+0000", 2024-04-25T01:35:32.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:35:32.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:35:32.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T01:35:32.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:35:32.282 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.823+0000 7f5b273a2700 1 Processor -- start 2024-04-25T01:35:32.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.823+0000 7f5b273a2700 1 -- start start 2024-04-25T01:35:32.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.824+0000 7f5b273a2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b20108630 0x7f5b2010aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:32.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.824+0000 7f5b273a2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2007bdd0 con 0x7f5b20108630 2024-04-25T01:35:32.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.824+0000 7f5b263a0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b20108630 0x7f5b2010aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:32.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.824+0000 7f5b263a0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b20108630 0x7f5b2010aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38034/0 (socket says 172.21.15.78:38034) 2024-04-25T01:35:32.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.824+0000 7f5b263a0700 1 -- 172.21.15.78:0/2056248566 learned_addr learned my addr 172.21.15.78:0/2056248566 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:32.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.824+0000 7f5b263a0700 1 -- 172.21.15.78:0/2056248566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2010af60 con 0x7f5b20108630 2024-04-25T01:35:32.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.824+0000 7f5b263a0700 1 --2- 172.21.15.78:0/2056248566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b20108630 0x7f5b2010aa20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0098a0 tx=0x7f5b1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27da0f561b1f256 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:32.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.824+0000 7f5b2539e700 1 -- 172.21.15.78:0/2056248566 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b1c004160 con 0x7f5b20108630 2024-04-25T01:35:32.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.825+0000 7f5b2539e700 1 -- 172.21.15.78:0/2056248566 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5b1c004300 con 0x7f5b20108630 2024-04-25T01:35:32.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.825+0000 7f5b2539e700 1 -- 172.21.15.78:0/2056248566 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b1c011b50 con 0x7f5b20108630 2024-04-25T01:35:32.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.825+0000 7f5b273a2700 1 -- 172.21.15.78:0/2056248566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b20108630 msgr2=0x7f5b2010aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:32.283 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.825+0000 7f5b273a2700 1 --2- 172.21.15.78:0/2056248566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b20108630 0x7f5b2010aa20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0098a0 tx=0x7f5b1c009bb0 comp rx=0 tx=0).stop 2024-04-25T01:35:32.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.825+0000 7f5b273a2700 1 -- 172.21.15.78:0/2056248566 shutdown_connections 2024-04-25T01:35:32.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.825+0000 7f5b273a2700 1 --2- 172.21.15.78:0/2056248566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b20108630 0x7f5b2010aa20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:32.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.825+0000 7f5b273a2700 1 -- 172.21.15.78:0/2056248566 >> 172.21.15.78:0/2056248566 conn(0x7f5b20101f80 msgr2=0x7f5b20104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:32.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.825+0000 7f5b273a2700 1 -- 172.21.15.78:0/2056248566 shutdown_connections 2024-04-25T01:35:32.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.825+0000 7f5b273a2700 1 -- 172.21.15.78:0/2056248566 wait complete. 2024-04-25T01:35:32.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.826+0000 7f5b273a2700 1 Processor -- start 2024-04-25T01:35:32.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.826+0000 7f5b273a2700 1 -- start start 2024-04-25T01:35:32.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.826+0000 7f5b273a2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b20108630 0x7f5b2019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:32.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.826+0000 7f5b273a2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2010bf20 con 0x7f5b20108630 2024-04-25T01:35:32.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.826+0000 7f5b263a0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b20108630 0x7f5b2019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:32.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.826+0000 7f5b263a0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b20108630 0x7f5b2019bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38042/0 (socket says 172.21.15.78:38042) 2024-04-25T01:35:32.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.826+0000 7f5b263a0700 1 -- 172.21.15.78:0/1034685946 learned_addr learned my addr 172.21.15.78:0/1034685946 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:32.284 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.826+0000 7f5b263a0700 1 -- 172.21.15.78:0/1034685946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2019c060 con 0x7f5b20108630 2024-04-25T01:35:32.285 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.827+0000 7f5b263a0700 1 --2- 172.21.15.78:0/1034685946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b20108630 0x7f5b2019bb20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c011b60 tx=0x7f5b1c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:32.285 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.827+0000 7f5b177fe700 1 -- 172.21.15.78:0/1034685946 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b1c003970 con 0x7f5b20108630 2024-04-25T01:35:32.285 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.827+0000 7f5b273a2700 1 -- 172.21.15.78:0/1034685946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2019c2f0 con 0x7f5b20108630 2024-04-25T01:35:32.285 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.827+0000 7f5b177fe700 1 -- 172.21.15.78:0/1034685946 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5b1c003b10 con 0x7f5b20108630 2024-04-25T01:35:32.285 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.827+0000 7f5b177fe700 1 -- 172.21.15.78:0/1034685946 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b1c024610 con 0x7f5b20108630 2024-04-25T01:35:32.285 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.827+0000 7f5b273a2700 1 -- 172.21.15.78:0/1034685946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2019efe0 con 0x7f5b20108630 2024-04-25T01:35:32.285 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.827+0000 7f5b177fe700 1 -- 172.21.15.78:0/1034685946 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5b1c024ca0 con 0x7f5b20108630 2024-04-25T01:35:32.285 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.827+0000 7f5b177fe700 1 -- 172.21.15.78:0/1034685946 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5b1c011e00 con 0x7f5b20108630 2024-04-25T01:35:32.285 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.828+0000 7f5b273a2700 1 -- 172.21.15.78:0/1034685946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b20063890 con 0x7f5b20108630 2024-04-25T01:35:32.285 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.832+0000 7f5b177fe700 1 -- 172.21.15.78:0/1034685946 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5b1c01a9f0 con 0x7f5b20108630 2024-04-25T01:35:32.285 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.903+0000 7f5b273a2700 1 -- 172.21.15.78:0/1034685946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5b2019f1b0 con 0x7f5b20108630 2024-04-25T01:35:32.285 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.904+0000 7f5b177fe700 1 -- 172.21.15.78:0/1034685946 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5b1c01a430 con 0x7f5b20108630 2024-04-25T01:35:32.286 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.905+0000 7f5b273a2700 1 -- 172.21.15.78:0/1034685946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b20108630 msgr2=0x7f5b2019bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:32.286 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.905+0000 7f5b273a2700 1 --2- 172.21.15.78:0/1034685946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b20108630 0x7f5b2019bb20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c011b60 tx=0x7f5b1c011c40 comp rx=0 tx=0).stop 2024-04-25T01:35:32.286 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.905+0000 7f5b273a2700 1 -- 172.21.15.78:0/1034685946 shutdown_connections 2024-04-25T01:35:32.286 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.905+0000 7f5b273a2700 1 --2- 172.21.15.78:0/1034685946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b20108630 0x7f5b2019bb20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:32.286 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.905+0000 7f5b273a2700 1 -- 172.21.15.78:0/1034685946 >> 172.21.15.78:0/1034685946 conn(0x7f5b20101f80 msgr2=0x7f5b20102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:32.286 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.905+0000 7f5b273a2700 1 -- 172.21.15.78:0/1034685946 shutdown_connections 2024-04-25T01:35:32.286 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:29.905+0000 7f5b273a2700 1 -- 172.21.15.78:0/1034685946 wait complete. 2024-04-25T01:35:32.286 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (1/15)... 2024-04-25T01:35:32.463 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:32 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:32.085+0000 7f22ba3b6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:35:32.463 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:32 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:32.290+0000 7f22ba3b6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:35:32.737 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:32 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:32.486+0000 7f22ba3b6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:35:33.331 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:33 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:33.092+0000 7f22ba3b6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T01:35:33.602 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:33 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:33.528+0000 7f22ba3b6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:35:33.602 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:33 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:33.626+0000 7f22ba3b6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:35:33.991 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:33 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:33.829+0000 7f22ba3b6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T01:35:34.672 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:34 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:34.362+0000 7f22ba3b6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:35:34.672 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:34 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:34.454+0000 7f22ba3b6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:35:34.993 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:34 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:34.695+0000 7f22ba3b6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:35:34.993 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:34 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:34.827+0000 7f22ba3b6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:35:34.994 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:34 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:34.921+0000 7f22ba3b6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:35:35.276 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:35 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:35.037+0000 7f22ba3b6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:35:35.276 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:35 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:35.133+0000 7f22ba3b6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:35:36.172 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:35 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:35.976+0000 7f22ba3b6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:35:36.173 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:36 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:36.196+0000 7f22ba3b6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:35:36.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:36 smithi078 ceph-mon[29469]: Activating manager daemon x 2024-04-25T01:35:36.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:36 smithi078 ceph-mon[29469]: mgrmap e2: x(active, starting, since 0.00314054s) 2024-04-25T01:35:36.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:36 smithi078 ceph-mon[29469]: from='mgr.14100 172.21.15.78:0/502233261' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:35:36.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:36 smithi078 ceph-mon[29469]: from='mgr.14100 172.21.15.78:0/502233261' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:35:36.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:36 smithi078 ceph-mon[29469]: from='mgr.14100 172.21.15.78:0/502233261' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:35:36.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:36 smithi078 ceph-mon[29469]: from='mgr.14100 172.21.15.78:0/502233261' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:35:36.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:36 smithi078 ceph-mon[29469]: from='mgr.14100 172.21.15.78:0/502233261' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T01:35:36.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:36 smithi078 ceph-mon[29469]: Manager daemon x is now available 2024-04-25T01:35:36.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:36 smithi078 ceph-mon[29469]: from='mgr.14100 172.21.15.78:0/502233261' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:35:36.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:36 smithi078 ceph-mon[29469]: from='mgr.14100 172.21.15.78:0/502233261' entity='mgr.x' 2024-04-25T01:35:36.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:36 smithi078 ceph-mon[29469]: from='mgr.14100 172.21.15.78:0/502233261' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T01:35:36.493 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:36 smithi078 ceph-mon[29469]: from='mgr.14100 172.21.15.78:0/502233261' entity='mgr.x' 2024-04-25T01:35:36.493 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:36 smithi078 ceph-mon[29469]: from='mgr.14100 172.21.15.78:0/502233261' entity='mgr.x' 2024-04-25T01:35:38.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:38 smithi078 ceph-mon[29469]: mgrmap e3: x(active, since 1.0075s) 2024-04-25T01:35:39.491 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:39 smithi078 ceph-mon[29469]: mgrmap e4: x(active, since 2s) 2024-04-25T01:35:39.491 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:39 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1709868768' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T01:35:41.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T01:35:41.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:35:41.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "df36bea8-02a3-11ef-bc93-c7b262605968", 2024-04-25T01:35:41.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T01:35:41.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T01:35:41.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T01:35:41.686 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T01:35:41.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:35:41.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T01:35:41.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T01:35:41.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-04-25T01:35:41.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:35:41.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T01:35:41.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T01:35:41.687 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:35:41.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-25T01:35:41.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T01:35:41.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:35:41.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T01:35:41.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T01:35:41.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:35:41.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T01:35:41.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:35:41.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T01:35:41.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T01:35:41.688 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T01:35:41.698 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T01:35:41.698 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T01:35:41.699 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T01:35:41.699 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:35:41.699 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T01:35:41.699 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T01:35:41.699 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T01:35:41.699 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T01:35:41.699 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T01:35:41.699 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T01:35:41.699 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T01:35:41.699 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T01:35:41.699 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T01:35:41.700 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:35:41.700 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T01:35:41.700 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:35:41.700 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T01:35:41.700 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T01:35:41.700 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:35:41.700 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T01:35:41.700 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T01:35:41.700 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T01:35:41.700 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T01:35:41.700 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T01:35:41.700 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T01:35:41.700 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T01:35:41.701 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:35:41.701 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:35:41.701 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:35:41.701 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T01:35:41.701 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:35:41.701 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T01:35:22.529621+0000", 2024-04-25T01:35:41.701 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:35:41.701 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:35:41.701 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T01:35:41.701 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:35:41.701 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.075+0000 7fc5ef301700 1 Processor -- start 2024-04-25T01:35:41.701 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.075+0000 7fc5ef301700 1 -- start start 2024-04-25T01:35:41.701 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.075+0000 7fc5ef301700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5e8107d60 0x7fc5e8108180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:41.702 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.075+0000 7fc5ef301700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e8108750 con 0x7fc5e8107d60 2024-04-25T01:35:41.702 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.076+0000 7fc5ed09d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5e8107d60 0x7fc5e8108180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:41.702 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.076+0000 7fc5ed09d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5e8107d60 0x7fc5e8108180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38130/0 (socket says 172.21.15.78:38130) 2024-04-25T01:35:41.702 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.076+0000 7fc5ed09d700 1 -- 172.21.15.78:0/3118354392 learned_addr learned my addr 172.21.15.78:0/3118354392 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:41.702 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.076+0000 7fc5ed09d700 1 -- 172.21.15.78:0/3118354392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e8108f10 con 0x7fc5e8107d60 2024-04-25T01:35:41.702 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.076+0000 7fc5ed09d700 1 --2- 172.21.15.78:0/3118354392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5e8107d60 0x7fc5e8108180 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40098a0 tx=0x7fc5e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ec894d6ea4b91a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:41.702 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.076+0000 7fc5dffff700 1 -- 172.21.15.78:0/3118354392 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5e4004160 con 0x7fc5e8107d60 2024-04-25T01:35:41.702 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.076+0000 7fc5dffff700 1 -- 172.21.15.78:0/3118354392 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc5e4004300 con 0x7fc5e8107d60 2024-04-25T01:35:41.702 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.076+0000 7fc5dffff700 1 -- 172.21.15.78:0/3118354392 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5e4011b50 con 0x7fc5e8107d60 2024-04-25T01:35:41.703 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.077+0000 7fc5ef301700 1 -- 172.21.15.78:0/3118354392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5e8107d60 msgr2=0x7fc5e8108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:41.703 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.077+0000 7fc5ef301700 1 --2- 172.21.15.78:0/3118354392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5e8107d60 0x7fc5e8108180 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40098a0 tx=0x7fc5e4009bb0 comp rx=0 tx=0).stop 2024-04-25T01:35:41.703 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.077+0000 7fc5ef301700 1 -- 172.21.15.78:0/3118354392 shutdown_connections 2024-04-25T01:35:41.703 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.077+0000 7fc5ef301700 1 --2- 172.21.15.78:0/3118354392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5e8107d60 0x7fc5e8108180 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:41.703 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.077+0000 7fc5ef301700 1 -- 172.21.15.78:0/3118354392 >> 172.21.15.78:0/3118354392 conn(0x7fc5e8103380 msgr2=0x7fc5e81057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:41.703 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.077+0000 7fc5ef301700 1 -- 172.21.15.78:0/3118354392 shutdown_connections 2024-04-25T01:35:41.703 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.077+0000 7fc5ef301700 1 -- 172.21.15.78:0/3118354392 wait complete. 2024-04-25T01:35:41.703 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.078+0000 7fc5ef301700 1 Processor -- start 2024-04-25T01:35:41.703 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.078+0000 7fc5ef301700 1 -- start start 2024-04-25T01:35:41.703 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.078+0000 7fc5ef301700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5e8107d60 0x7fc5e819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:41.703 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.078+0000 7fc5ef301700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e810c2c0 con 0x7fc5e8107d60 2024-04-25T01:35:41.703 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.078+0000 7fc5ed09d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5e8107d60 0x7fc5e819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:41.703 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.078+0000 7fc5ed09d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5e8107d60 0x7fc5e819bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38136/0 (socket says 172.21.15.78:38136) 2024-04-25T01:35:41.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.078+0000 7fc5ed09d700 1 -- 172.21.15.78:0/1709868768 learned_addr learned my addr 172.21.15.78:0/1709868768 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:41.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.078+0000 7fc5ed09d700 1 -- 172.21.15.78:0/1709868768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e819c1c0 con 0x7fc5e8107d60 2024-04-25T01:35:41.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.078+0000 7fc5ed09d700 1 --2- 172.21.15.78:0/1709868768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5e8107d60 0x7fc5e819bc80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4011670 tx=0x7fc5e4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:41.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.078+0000 7fc5de7fc700 1 -- 172.21.15.78:0/1709868768 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5e40039c0 con 0x7fc5e8107d60 2024-04-25T01:35:41.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.079+0000 7fc5de7fc700 1 -- 172.21.15.78:0/1709868768 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc5e4011e50 con 0x7fc5e8107d60 2024-04-25T01:35:41.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.079+0000 7fc5ef301700 1 -- 172.21.15.78:0/1709868768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e819c450 con 0x7fc5e8107d60 2024-04-25T01:35:41.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.079+0000 7fc5de7fc700 1 -- 172.21.15.78:0/1709868768 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5e401a5c0 con 0x7fc5e8107d60 2024-04-25T01:35:41.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.079+0000 7fc5ef301700 1 -- 172.21.15.78:0/1709868768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e819f140 con 0x7fc5e8107d60 2024-04-25T01:35:41.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.079+0000 7fc5de7fc700 1 -- 172.21.15.78:0/1709868768 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fc5e4024430 con 0x7fc5e8107d60 2024-04-25T01:35:41.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.079+0000 7fc5ef301700 1 -- 172.21.15.78:0/1709868768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e8063890 con 0x7fc5e8107d60 2024-04-25T01:35:41.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.080+0000 7fc5de7fc700 1 --2- 172.21.15.78:0/1709868768 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fc5d4038ea0 0x7fc5d403b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:41.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.080+0000 7fc5de7fc700 1 -- 172.21.15.78:0/1709868768 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc5e404d270 con 0x7fc5e8107d60 2024-04-25T01:35:41.704 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.080+0000 7fc5ec89c700 1 --2- 172.21.15.78:0/1709868768 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fc5d4038ea0 0x7fc5d403b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.080+0000 7fc5ec89c700 1 --2- 172.21.15.78:0/1709868768 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fc5d4038ea0 0x7fc5d403b360 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8006fd0 tx=0x7fc5d8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.085+0000 7fc5de7fc700 1 -- 172.21.15.78:0/1709868768 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc5e402d910 con 0x7fc5e8107d60 2024-04-25T01:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.205+0000 7fc5de7fc700 1 -- 172.21.15.78:0/1709868768 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fc5e402d3f0 con 0x7fc5e8107d60 2024-04-25T01:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.365+0000 7fc5ef301700 1 -- 172.21.15.78:0/1709868768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc5e819f310 con 0x7fc5e8107d60 2024-04-25T01:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.365+0000 7fc5de7fc700 1 -- 172.21.15.78:0/1709868768 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc5e819f310 con 0x7fc5e8107d60 2024-04-25T01:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.367+0000 7fc5ef301700 1 -- 172.21.15.78:0/1709868768 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fc5d4038ea0 msgr2=0x7fc5d403b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.367+0000 7fc5ef301700 1 --2- 172.21.15.78:0/1709868768 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fc5d4038ea0 0x7fc5d403b360 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8006fd0 tx=0x7fc5d8006e40 comp rx=0 tx=0).stop 2024-04-25T01:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.367+0000 7fc5ef301700 1 -- 172.21.15.78:0/1709868768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5e8107d60 msgr2=0x7fc5e819bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.367+0000 7fc5ef301700 1 --2- 172.21.15.78:0/1709868768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5e8107d60 0x7fc5e819bc80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4011670 tx=0x7fc5e4011b60 comp rx=0 tx=0).stop 2024-04-25T01:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.367+0000 7fc5ef301700 1 -- 172.21.15.78:0/1709868768 shutdown_connections 2024-04-25T01:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.367+0000 7fc5ef301700 1 --2- 172.21.15.78:0/1709868768 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fc5d4038ea0 0x7fc5d403b360 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.367+0000 7fc5ef301700 1 --2- 172.21.15.78:0/1709868768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5e8107d60 0x7fc5e819bc80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.367+0000 7fc5ef301700 1 -- 172.21.15.78:0/1709868768 >> 172.21.15.78:0/1709868768 conn(0x7fc5e8103380 msgr2=0x7fc5e8103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:41.705 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.367+0000 7fc5ef301700 1 -- 172.21.15.78:0/1709868768 shutdown_connections 2024-04-25T01:35:41.706 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:38.367+0000 7fc5ef301700 1 -- 172.21.15.78:0/1709868768 wait complete. 2024-04-25T01:35:41.706 INFO:teuthology.orchestra.run.smithi078.stdout:mgr is available 2024-04-25T01:35:42.652 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:42 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/58284058' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T01:35:42.895 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T01:35:42.895 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T01:35:42.895 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout fsid = df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:35:42.895 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T01:35:42.895 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T01:35:42.895 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T01:35:42.895 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T01:35:42.895 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T01:35:42.895 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T01:35:42.895 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T01:35:42.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-04-25T01:35:42.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T01:35:42.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T01:35:42.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T01:35:42.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T01:35:42.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T01:35:42.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T01:35:42.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.167+0000 7fa60b62f700 1 Processor -- start 2024-04-25T01:35:42.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.167+0000 7fa60b62f700 1 -- start start 2024-04-25T01:35:42.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.167+0000 7fa60b62f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6041081f0 0x7fa604108610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:42.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.167+0000 7fa60b62f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa604108be0 con 0x7fa6041081f0 2024-04-25T01:35:42.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.167+0000 7fa6093cb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6041081f0 0x7fa604108610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:42.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.167+0000 7fa6093cb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6041081f0 0x7fa604108610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33544/0 (socket says 172.21.15.78:33544) 2024-04-25T01:35:42.896 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.167+0000 7fa6093cb700 1 -- 172.21.15.78:0/1382528207 learned_addr learned my addr 172.21.15.78:0/1382528207 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:42.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.168+0000 7fa6093cb700 1 -- 172.21.15.78:0/1382528207 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa604106e00 con 0x7fa6041081f0 2024-04-25T01:35:42.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.168+0000 7fa6093cb700 1 --2- 172.21.15.78:0/1382528207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6041081f0 0x7fa604108610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa6000098a0 tx=0x7fa600009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36bf8b5bd7e1de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:42.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.168+0000 7fa5fbfff700 1 -- 172.21.15.78:0/1382528207 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa600004160 con 0x7fa6041081f0 2024-04-25T01:35:42.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.168+0000 7fa5fbfff700 1 -- 172.21.15.78:0/1382528207 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa600004300 con 0x7fa6041081f0 2024-04-25T01:35:42.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.168+0000 7fa5fbfff700 1 -- 172.21.15.78:0/1382528207 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa600011b50 con 0x7fa6041081f0 2024-04-25T01:35:42.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.168+0000 7fa60b62f700 1 -- 172.21.15.78:0/1382528207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6041081f0 msgr2=0x7fa604108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:42.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.168+0000 7fa60b62f700 1 --2- 172.21.15.78:0/1382528207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6041081f0 0x7fa604108610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa6000098a0 tx=0x7fa600009bb0 comp rx=0 tx=0).stop 2024-04-25T01:35:42.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.169+0000 7fa60b62f700 1 -- 172.21.15.78:0/1382528207 shutdown_connections 2024-04-25T01:35:42.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.169+0000 7fa60b62f700 1 --2- 172.21.15.78:0/1382528207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6041081f0 0x7fa604108610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:42.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.169+0000 7fa60b62f700 1 -- 172.21.15.78:0/1382528207 >> 172.21.15.78:0/1382528207 conn(0x7fa6041018f0 msgr2=0x7fa604103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:42.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.169+0000 7fa60b62f700 1 -- 172.21.15.78:0/1382528207 shutdown_connections 2024-04-25T01:35:42.897 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.169+0000 7fa60b62f700 1 -- 172.21.15.78:0/1382528207 wait complete. 2024-04-25T01:35:42.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.169+0000 7fa60b62f700 1 Processor -- start 2024-04-25T01:35:42.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.169+0000 7fa60b62f700 1 -- start start 2024-04-25T01:35:42.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.169+0000 7fa60b62f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6041081f0 0x7fa60419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:42.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.169+0000 7fa60b62f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa60410b1c0 con 0x7fa6041081f0 2024-04-25T01:35:42.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.170+0000 7fa6093cb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6041081f0 0x7fa60419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:42.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.170+0000 7fa6093cb700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6041081f0 0x7fa60419bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33550/0 (socket says 172.21.15.78:33550) 2024-04-25T01:35:42.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.170+0000 7fa6093cb700 1 -- 172.21.15.78:0/58284058 learned_addr learned my addr 172.21.15.78:0/58284058 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:42.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.170+0000 7fa6093cb700 1 -- 172.21.15.78:0/58284058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60419c1e0 con 0x7fa6041081f0 2024-04-25T01:35:42.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.170+0000 7fa6093cb700 1 --2- 172.21.15.78:0/58284058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6041081f0 0x7fa60419bca0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa600011670 tx=0x7fa600011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:42.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.170+0000 7fa5fa7fc700 1 -- 172.21.15.78:0/58284058 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6000039c0 con 0x7fa6041081f0 2024-04-25T01:35:42.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.170+0000 7fa5fa7fc700 1 -- 172.21.15.78:0/58284058 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa600011e50 con 0x7fa6041081f0 2024-04-25T01:35:42.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.170+0000 7fa60b62f700 1 -- 172.21.15.78:0/58284058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60419c470 con 0x7fa6041081f0 2024-04-25T01:35:42.898 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.170+0000 7fa5fa7fc700 1 -- 172.21.15.78:0/58284058 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa60001a5c0 con 0x7fa6041081f0 2024-04-25T01:35:42.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.170+0000 7fa60b62f700 1 -- 172.21.15.78:0/58284058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60419f160 con 0x7fa6041081f0 2024-04-25T01:35:42.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.171+0000 7fa5fa7fc700 1 -- 172.21.15.78:0/58284058 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fa60001a870 con 0x7fa6041081f0 2024-04-25T01:35:42.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.171+0000 7fa60b62f700 1 -- 172.21.15.78:0/58284058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa604063890 con 0x7fa6041081f0 2024-04-25T01:35:42.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.171+0000 7fa5fa7fc700 1 --2- 172.21.15.78:0/58284058 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fa5f0039000 0x7fa5f003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:42.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.171+0000 7fa5fa7fc700 1 -- 172.21.15.78:0/58284058 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa60004d520 con 0x7fa6041081f0 2024-04-25T01:35:42.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.172+0000 7fa608bca700 1 --2- 172.21.15.78:0/58284058 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fa5f0039000 0x7fa5f003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:42.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.172+0000 7fa608bca700 1 --2- 172.21.15.78:0/58284058 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fa5f0039000 0x7fa5f003b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4006fd0 tx=0x7fa5f4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:42.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.176+0000 7fa5fa7fc700 1 -- 172.21.15.78:0/58284058 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa600019c70 con 0x7fa6041081f0 2024-04-25T01:35:42.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.373+0000 7fa60b62f700 1 -- 172.21.15.78:0/58284058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa6040008d0 con 0x7fa6041081f0 2024-04-25T01:35:42.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.374+0000 7fa5fa7fc700 1 -- 172.21.15.78:0/58284058 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fa60004cf40 con 0x7fa6041081f0 2024-04-25T01:35:42.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.376+0000 7fa60b62f700 1 -- 172.21.15.78:0/58284058 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fa5f0039000 msgr2=0x7fa5f003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:42.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.376+0000 7fa60b62f700 1 --2- 172.21.15.78:0/58284058 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fa5f0039000 0x7fa5f003b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4006fd0 tx=0x7fa5f4006e40 comp rx=0 tx=0).stop 2024-04-25T01:35:42.899 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.376+0000 7fa60b62f700 1 -- 172.21.15.78:0/58284058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6041081f0 msgr2=0x7fa60419bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:42.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.376+0000 7fa60b62f700 1 --2- 172.21.15.78:0/58284058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6041081f0 0x7fa60419bca0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa600011670 tx=0x7fa600011b60 comp rx=0 tx=0).stop 2024-04-25T01:35:42.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.376+0000 7fa60b62f700 1 -- 172.21.15.78:0/58284058 shutdown_connections 2024-04-25T01:35:42.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.376+0000 7fa60b62f700 1 --2- 172.21.15.78:0/58284058 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fa5f0039000 0x7fa5f003b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:42.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.376+0000 7fa60b62f700 1 --2- 172.21.15.78:0/58284058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6041081f0 0x7fa60419bca0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:42.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.376+0000 7fa60b62f700 1 -- 172.21.15.78:0/58284058 >> 172.21.15.78:0/58284058 conn(0x7fa6041018f0 msgr2=0x7fa6041027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:42.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.376+0000 7fa60b62f700 1 -- 172.21.15.78:0/58284058 shutdown_connections 2024-04-25T01:35:42.900 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:42.376+0000 7fa60b62f700 1 -- 172.21.15.78:0/58284058 wait complete. 2024-04-25T01:35:42.900 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling cephadm module... 2024-04-25T01:35:43.949 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:43 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3516641214' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T01:35:43.949 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:43 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: ignoring --setuser ceph since I am not root 2024-04-25T01:35:43.949 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:43 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: ignoring --setgroup ceph since I am not root 2024-04-25T01:35:43.949 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:43 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:43.790+0000 7f4675f68200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:35:43.949 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:43 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:43.972+0000 7f4675f68200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:35:44.159 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.316+0000 7fe880649700 1 Processor -- start 2024-04-25T01:35:44.159 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.316+0000 7fe880649700 1 -- start start 2024-04-25T01:35:44.159 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.316+0000 7fe880649700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781087a0 0x7fe87810ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:44.159 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.316+0000 7fe880649700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe87807c670 con 0x7fe8781087a0 2024-04-25T01:35:44.159 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.316+0000 7fe87e3e5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781087a0 0x7fe87810ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:44.159 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.316+0000 7fe87e3e5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781087a0 0x7fe87810ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33556/0 (socket says 172.21.15.78:33556) 2024-04-25T01:35:44.159 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.316+0000 7fe87e3e5700 1 -- 172.21.15.78:0/3547035974 learned_addr learned my addr 172.21.15.78:0/3547035974 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:44.160 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.317+0000 7fe87e3e5700 1 -- 172.21.15.78:0/3547035974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87810b0d0 con 0x7fe8781087a0 2024-04-25T01:35:44.160 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.317+0000 7fe87e3e5700 1 --2- 172.21.15.78:0/3547035974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781087a0 0x7fe87810ab90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe8740098a0 tx=0x7fe874009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f681568570f260 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:44.160 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.317+0000 7fe87d3e3700 1 -- 172.21.15.78:0/3547035974 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe874004160 con 0x7fe8781087a0 2024-04-25T01:35:44.160 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.317+0000 7fe87d3e3700 1 -- 172.21.15.78:0/3547035974 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe874004300 con 0x7fe8781087a0 2024-04-25T01:35:44.160 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.317+0000 7fe87d3e3700 1 -- 172.21.15.78:0/3547035974 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe874011b50 con 0x7fe8781087a0 2024-04-25T01:35:44.160 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.317+0000 7fe880649700 1 -- 172.21.15.78:0/3547035974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781087a0 msgr2=0x7fe87810ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:44.160 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.318+0000 7fe880649700 1 --2- 172.21.15.78:0/3547035974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781087a0 0x7fe87810ab90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe8740098a0 tx=0x7fe874009bb0 comp rx=0 tx=0).stop 2024-04-25T01:35:44.160 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.318+0000 7fe880649700 1 -- 172.21.15.78:0/3547035974 shutdown_connections 2024-04-25T01:35:44.161 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.318+0000 7fe880649700 1 --2- 172.21.15.78:0/3547035974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781087a0 0x7fe87810ab90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:44.161 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.318+0000 7fe880649700 1 -- 172.21.15.78:0/3547035974 >> 172.21.15.78:0/3547035974 conn(0x7fe8781021d0 msgr2=0x7fe8781045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:44.161 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.318+0000 7fe880649700 1 -- 172.21.15.78:0/3547035974 shutdown_connections 2024-04-25T01:35:44.161 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.318+0000 7fe880649700 1 -- 172.21.15.78:0/3547035974 wait complete. 2024-04-25T01:35:44.161 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.318+0000 7fe880649700 1 Processor -- start 2024-04-25T01:35:44.161 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.318+0000 7fe880649700 1 -- start start 2024-04-25T01:35:44.161 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.318+0000 7fe880649700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781087a0 0x7fe87819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:44.161 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.318+0000 7fe880649700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe87810c090 con 0x7fe8781087a0 2024-04-25T01:35:44.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.319+0000 7fe87e3e5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781087a0 0x7fe87819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:44.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.319+0000 7fe87e3e5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781087a0 0x7fe87819bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33566/0 (socket says 172.21.15.78:33566) 2024-04-25T01:35:44.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.319+0000 7fe87e3e5700 1 -- 172.21.15.78:0/3516641214 learned_addr learned my addr 172.21.15.78:0/3516641214 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:44.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.319+0000 7fe87e3e5700 1 -- 172.21.15.78:0/3516641214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87819c130 con 0x7fe8781087a0 2024-04-25T01:35:44.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.319+0000 7fe87e3e5700 1 --2- 172.21.15.78:0/3516641214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781087a0 0x7fe87819bbf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe874011670 tx=0x7fe874011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:44.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.319+0000 7fe86f7fe700 1 -- 172.21.15.78:0/3516641214 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8740039c0 con 0x7fe8781087a0 2024-04-25T01:35:44.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.319+0000 7fe86f7fe700 1 -- 172.21.15.78:0/3516641214 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe874011e50 con 0x7fe8781087a0 2024-04-25T01:35:44.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.319+0000 7fe880649700 1 -- 172.21.15.78:0/3516641214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe87819c3c0 con 0x7fe8781087a0 2024-04-25T01:35:44.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.319+0000 7fe86f7fe700 1 -- 172.21.15.78:0/3516641214 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe87401a5c0 con 0x7fe8781087a0 2024-04-25T01:35:44.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.319+0000 7fe880649700 1 -- 172.21.15.78:0/3516641214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe87819f0b0 con 0x7fe8781087a0 2024-04-25T01:35:44.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.320+0000 7fe86f7fe700 1 -- 172.21.15.78:0/3516641214 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fe87401a870 con 0x7fe8781087a0 2024-04-25T01:35:44.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.320+0000 7fe86f7fe700 1 --2- 172.21.15.78:0/3516641214 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fe864039000 0x7fe86403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:44.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.320+0000 7fe880649700 1 -- 172.21.15.78:0/3516641214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe878063890 con 0x7fe8781087a0 2024-04-25T01:35:44.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.320+0000 7fe86f7fe700 1 -- 172.21.15.78:0/3516641214 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe87404d230 con 0x7fe8781087a0 2024-04-25T01:35:44.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.320+0000 7fe87dbe4700 1 --2- 172.21.15.78:0/3516641214 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fe864039000 0x7fe86403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:44.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.321+0000 7fe87dbe4700 1 --2- 172.21.15.78:0/3516641214 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fe864039000 0x7fe86403b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe868006fd0 tx=0x7fe868006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:44.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.325+0000 7fe86f7fe700 1 -- 172.21.15.78:0/3516641214 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe874017ca0 con 0x7fe8781087a0 2024-04-25T01:35:44.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.573+0000 7fe880649700 1 -- 172.21.15.78:0/3516641214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fe87819f280 con 0x7fe8781087a0 2024-04-25T01:35:44.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.624+0000 7fe86f7fe700 1 -- 172.21.15.78:0/3516641214 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fe87819f280 con 0x7fe8781087a0 2024-04-25T01:35:44.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.626+0000 7fe86f7fe700 1 -- 172.21.15.78:0/3516641214 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fe87402d910 con 0x7fe8781087a0 2024-04-25T01:35:44.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.627+0000 7fe880649700 1 -- 172.21.15.78:0/3516641214 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fe864039000 msgr2=0x7fe86403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:44.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.627+0000 7fe880649700 1 --2- 172.21.15.78:0/3516641214 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fe864039000 0x7fe86403b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe868006fd0 tx=0x7fe868006e40 comp rx=0 tx=0).stop 2024-04-25T01:35:44.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.627+0000 7fe880649700 1 -- 172.21.15.78:0/3516641214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781087a0 msgr2=0x7fe87819bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:44.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.627+0000 7fe880649700 1 --2- 172.21.15.78:0/3516641214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781087a0 0x7fe87819bbf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe874011670 tx=0x7fe874011b60 comp rx=0 tx=0).stop 2024-04-25T01:35:44.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.627+0000 7fe880649700 1 -- 172.21.15.78:0/3516641214 shutdown_connections 2024-04-25T01:35:44.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.627+0000 7fe880649700 1 --2- 172.21.15.78:0/3516641214 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fe864039000 0x7fe86403b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:44.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.627+0000 7fe880649700 1 --2- 172.21.15.78:0/3516641214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8781087a0 0x7fe87819bbf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:44.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.627+0000 7fe880649700 1 -- 172.21.15.78:0/3516641214 >> 172.21.15.78:0/3516641214 conn(0x7fe8781021d0 msgr2=0x7fe878102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:44.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.627+0000 7fe880649700 1 -- 172.21.15.78:0/3516641214 shutdown_connections 2024-04-25T01:35:44.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:43.627+0000 7fe880649700 1 -- 172.21.15.78:0/3516641214 wait complete. 2024-04-25T01:35:44.241 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:44 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:44.075+0000 7f4675f68200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:35:44.544 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:44 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:44.391+0000 7f4675f68200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:35:44.545 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:44 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:44.476+0000 7f4675f68200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:35:44.842 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:44 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3516641214' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T01:35:44.842 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:44 smithi078 ceph-mon[29469]: mgrmap e5: x(active, since 7s) 2024-04-25T01:35:45.443 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:35:45.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T01:35:45.444 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T01:35:45.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T01:35:45.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T01:35:45.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:35:45.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.583+0000 7fea8d783700 1 Processor -- start 2024-04-25T01:35:45.445 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.583+0000 7fea8d783700 1 -- start start 2024-04-25T01:35:45.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.583+0000 7fea8d783700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea88108810 0x7fea8810ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:45.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.583+0000 7fea8d783700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea8807c5f0 con 0x7fea88108810 2024-04-25T01:35:45.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.583+0000 7fea86ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea88108810 0x7fea8810ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:45.446 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.583+0000 7fea86ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea88108810 0x7fea8810ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33588/0 (socket says 172.21.15.78:33588) 2024-04-25T01:35:45.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.583+0000 7fea86ffd700 1 -- 172.21.15.78:0/528934920 learned_addr learned my addr 172.21.15.78:0/528934920 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:45.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.584+0000 7fea86ffd700 1 -- 172.21.15.78:0/528934920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8810b140 con 0x7fea88108810 2024-04-25T01:35:45.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.584+0000 7fea86ffd700 1 --2- 172.21.15.78:0/528934920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea88108810 0x7fea8810ac00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fea780098a0 tx=0x7fea78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cc383ab5d13dd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:45.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.584+0000 7fea85ffb700 1 -- 172.21.15.78:0/528934920 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea78004160 con 0x7fea88108810 2024-04-25T01:35:45.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.584+0000 7fea85ffb700 1 -- 172.21.15.78:0/528934920 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea78004300 con 0x7fea88108810 2024-04-25T01:35:45.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.584+0000 7fea85ffb700 1 -- 172.21.15.78:0/528934920 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea78011b50 con 0x7fea88108810 2024-04-25T01:35:45.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.585+0000 7fea8d783700 1 -- 172.21.15.78:0/528934920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea88108810 msgr2=0x7fea8810ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:45.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.585+0000 7fea8d783700 1 --2- 172.21.15.78:0/528934920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea88108810 0x7fea8810ac00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fea780098a0 tx=0x7fea78009bb0 comp rx=0 tx=0).stop 2024-04-25T01:35:45.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.585+0000 7fea8d783700 1 -- 172.21.15.78:0/528934920 shutdown_connections 2024-04-25T01:35:45.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.585+0000 7fea8d783700 1 --2- 172.21.15.78:0/528934920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea88108810 0x7fea8810ac00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:45.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.585+0000 7fea8d783700 1 -- 172.21.15.78:0/528934920 >> 172.21.15.78:0/528934920 conn(0x7fea881021a0 msgr2=0x7fea881045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:45.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.585+0000 7fea8d783700 1 -- 172.21.15.78:0/528934920 shutdown_connections 2024-04-25T01:35:45.447 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.585+0000 7fea8d783700 1 -- 172.21.15.78:0/528934920 wait complete. 2024-04-25T01:35:45.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.585+0000 7fea8d783700 1 Processor -- start 2024-04-25T01:35:45.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.586+0000 7fea8d783700 1 -- start start 2024-04-25T01:35:45.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.586+0000 7fea8d783700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea88108810 0x7fea8819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:45.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.586+0000 7fea8d783700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea8810c0a0 con 0x7fea88108810 2024-04-25T01:35:45.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.586+0000 7fea86ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea88108810 0x7fea8819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:45.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.586+0000 7fea86ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea88108810 0x7fea8819bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33594/0 (socket says 172.21.15.78:33594) 2024-04-25T01:35:45.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.586+0000 7fea86ffd700 1 -- 172.21.15.78:0/767869017 learned_addr learned my addr 172.21.15.78:0/767869017 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:45.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.586+0000 7fea86ffd700 1 -- 172.21.15.78:0/767869017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8819c140 con 0x7fea88108810 2024-04-25T01:35:45.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.586+0000 7fea86ffd700 1 --2- 172.21.15.78:0/767869017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea88108810 0x7fea8819bc00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fea78011b60 tx=0x7fea78011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:45.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.587+0000 7fea7ffff700 1 -- 172.21.15.78:0/767869017 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea78003b20 con 0x7fea88108810 2024-04-25T01:35:45.448 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.587+0000 7fea8d783700 1 -- 172.21.15.78:0/767869017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8819c3d0 con 0x7fea88108810 2024-04-25T01:35:45.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.587+0000 7fea7ffff700 1 -- 172.21.15.78:0/767869017 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea78003cc0 con 0x7fea88108810 2024-04-25T01:35:45.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.587+0000 7fea7ffff700 1 -- 172.21.15.78:0/767869017 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea78024840 con 0x7fea88108810 2024-04-25T01:35:45.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.587+0000 7fea8d783700 1 -- 172.21.15.78:0/767869017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8819f0c0 con 0x7fea88108810 2024-04-25T01:35:45.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.587+0000 7fea7ffff700 1 -- 172.21.15.78:0/767869017 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fea78022020 con 0x7fea88108810 2024-04-25T01:35:45.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.588+0000 7fea7ffff700 1 --2- 172.21.15.78:0/767869017 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fea74039050 0x7fea7403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:45.449 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.588+0000 7fea7ffff700 1 -- 172.21.15.78:0/767869017 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fea78011e00 con 0x7fea88108810 2024-04-25T01:35:45.450 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.588+0000 7fea867fc700 1 -- 172.21.15.78:0/767869017 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fea74039050 msgr2=0x7fea7403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1544860340 2024-04-25T01:35:45.450 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.588+0000 7fea867fc700 1 --2- 172.21.15.78:0/767869017 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fea74039050 0x7fea7403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:45.450 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.588+0000 7fea8d783700 1 -- 172.21.15.78:0/767869017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea88063890 con 0x7fea88108810 2024-04-25T01:35:45.450 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.597+0000 7fea7ffff700 1 -- 172.21.15.78:0/767869017 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fea78019c70 con 0x7fea88108810 2024-04-25T01:35:45.450 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.789+0000 7fea867fc700 1 -- 172.21.15.78:0/767869017 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fea74039050 msgr2=0x7fea7403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1544860340 2024-04-25T01:35:45.450 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.789+0000 7fea867fc700 1 --2- 172.21.15.78:0/767869017 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fea74039050 0x7fea7403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:35:45.450 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.865+0000 7fea8d783700 1 -- 172.21.15.78:0/767869017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fea8819f290 con 0x7fea88108810 2024-04-25T01:35:45.450 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.867+0000 7fea7ffff700 1 -- 172.21.15.78:0/767869017 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fea7804cb70 con 0x7fea88108810 2024-04-25T01:35:45.451 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.870+0000 7fea8d783700 1 -- 172.21.15.78:0/767869017 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fea74039050 msgr2=0x7fea7403b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:35:45.451 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.870+0000 7fea8d783700 1 --2- 172.21.15.78:0/767869017 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fea74039050 0x7fea7403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:45.451 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.870+0000 7fea8d783700 1 -- 172.21.15.78:0/767869017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea88108810 msgr2=0x7fea8819bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:45.451 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.870+0000 7fea8d783700 1 --2- 172.21.15.78:0/767869017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea88108810 0x7fea8819bc00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fea78011b60 tx=0x7fea78011c40 comp rx=0 tx=0).stop 2024-04-25T01:35:45.451 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.871+0000 7fea8d783700 1 -- 172.21.15.78:0/767869017 shutdown_connections 2024-04-25T01:35:45.451 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.871+0000 7fea8d783700 1 --2- 172.21.15.78:0/767869017 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7fea74039050 0x7fea7403b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:45.451 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.871+0000 7fea8d783700 1 --2- 172.21.15.78:0/767869017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea88108810 0x7fea8819bc00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:45.451 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.871+0000 7fea8d783700 1 -- 172.21.15.78:0/767869017 >> 172.21.15.78:0/767869017 conn(0x7fea881021a0 msgr2=0x7fea88102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:45.451 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.876+0000 7fea8d783700 1 -- 172.21.15.78:0/767869017 shutdown_connections 2024-04-25T01:35:45.451 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:44.876+0000 7fea8d783700 1 -- 172.21.15.78:0/767869017 wait complete. 2024-04-25T01:35:45.451 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for the mgr to restart... 2024-04-25T01:35:45.451 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr epoch 5... 2024-04-25T01:35:45.695 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:45 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/767869017' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T01:35:45.991 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:45 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:45.719+0000 7f4675f68200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:35:45.991 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:45 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:45.897+0000 7f4675f68200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:35:46.297 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:46 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:46.068+0000 7f4675f68200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:35:46.633 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:46 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:46.362+0000 7f4675f68200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:35:46.633 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:46 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:46.454+0000 7f4675f68200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:35:46.634 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:46 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:46.657+0000 7f4675f68200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:35:46.991 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:46 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:46.849+0000 7f4675f68200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:35:47.741 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:47 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:47.452+0000 7f4675f68200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T01:35:48.156 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:47 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:47.879+0000 7f4675f68200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:35:48.156 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:47 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:47.976+0000 7f4675f68200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:35:48.156 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:48 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:48.180+0000 7f4675f68200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T01:35:48.991 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:48 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:48.720+0000 7f4675f68200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:35:48.991 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:48 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:48.811+0000 7f4675f68200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:35:49.375 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:49 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:49.055+0000 7f4675f68200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:35:49.375 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:49 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:49.190+0000 7f4675f68200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:35:49.375 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:49 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:49.284+0000 7f4675f68200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:35:49.375 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:49 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:49.398+0000 7f4675f68200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:35:49.741 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:49 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:49.490+0000 7f4675f68200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:35:50.534 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:50.291+0000 7f4675f68200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:35:50.534 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:35:50.505+0000 7f4675f68200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:35:50.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: Active manager daemon x restarted 2024-04-25T01:35:50.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: Activating manager daemon x 2024-04-25T01:35:50.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T01:35:50.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: mgrmap e6: x(active, starting, since 0.00373837s) 2024-04-25T01:35:50.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:35:50.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T01:35:50.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:35:50.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:35:50.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:35:50.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: Manager daemon x is now available 2024-04-25T01:35:50.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' 2024-04-25T01:35:50.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' 2024-04-25T01:35:50.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:35:50.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:35:50.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:50 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:35:51.741 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:51 smithi078 ceph-mon[29469]: Found migration_current of "None". Setting to last migration. 2024-04-25T01:35:51.741 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:51 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T01:35:51.741 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:51 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' 2024-04-25T01:35:51.742 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:51 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' 2024-04-25T01:35:51.742 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:51 smithi078 ceph-mon[29469]: mgrmap e7: x(active, since 1.00633s) 2024-04-25T01:35:52.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:35:52.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T01:35:52.090 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T01:35:52.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:35:52.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.907+0000 7ff34c1f8700 1 Processor -- start 2024-04-25T01:35:52.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.907+0000 7ff34c1f8700 1 -- start start 2024-04-25T01:35:52.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.907+0000 7ff34c1f8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff344108790 0x7ff34410ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:52.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.908+0000 7ff34c1f8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff34407c670 con 0x7ff344108790 2024-04-25T01:35:52.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.908+0000 7ff349f94700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff344108790 0x7ff34410ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:52.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.908+0000 7ff349f94700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff344108790 0x7ff34410ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33600/0 (socket says 172.21.15.78:33600) 2024-04-25T01:35:52.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.908+0000 7ff349f94700 1 -- 172.21.15.78:0/2362367508 learned_addr learned my addr 172.21.15.78:0/2362367508 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:52.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.908+0000 7ff349f94700 1 -- 172.21.15.78:0/2362367508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34410b0c0 con 0x7ff344108790 2024-04-25T01:35:52.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.908+0000 7ff349f94700 1 --2- 172.21.15.78:0/2362367508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff344108790 0x7ff34410ab80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff3340098a0 tx=0x7ff334009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debbabbe6b259586 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:52.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.908+0000 7ff348f92700 1 -- 172.21.15.78:0/2362367508 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff334004160 con 0x7ff344108790 2024-04-25T01:35:52.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.908+0000 7ff348f92700 1 -- 172.21.15.78:0/2362367508 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff334004300 con 0x7ff344108790 2024-04-25T01:35:52.091 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.908+0000 7ff348f92700 1 -- 172.21.15.78:0/2362367508 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff334011b50 con 0x7ff344108790 2024-04-25T01:35:52.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.909+0000 7ff34c1f8700 1 -- 172.21.15.78:0/2362367508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff344108790 msgr2=0x7ff34410ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:52.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.909+0000 7ff34c1f8700 1 --2- 172.21.15.78:0/2362367508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff344108790 0x7ff34410ab80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff3340098a0 tx=0x7ff334009bb0 comp rx=0 tx=0).stop 2024-04-25T01:35:52.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.909+0000 7ff34c1f8700 1 -- 172.21.15.78:0/2362367508 shutdown_connections 2024-04-25T01:35:52.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.909+0000 7ff34c1f8700 1 --2- 172.21.15.78:0/2362367508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff344108790 0x7ff34410ab80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:52.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.909+0000 7ff34c1f8700 1 -- 172.21.15.78:0/2362367508 >> 172.21.15.78:0/2362367508 conn(0x7ff3441021a0 msgr2=0x7ff3441045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:52.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.909+0000 7ff34c1f8700 1 -- 172.21.15.78:0/2362367508 shutdown_connections 2024-04-25T01:35:52.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.909+0000 7ff34c1f8700 1 -- 172.21.15.78:0/2362367508 wait complete. 2024-04-25T01:35:52.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.910+0000 7ff34c1f8700 1 Processor -- start 2024-04-25T01:35:52.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.910+0000 7ff34c1f8700 1 -- start start 2024-04-25T01:35:52.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.910+0000 7ff34c1f8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff344108790 0x7ff34419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:52.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.910+0000 7ff34c1f8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff34410c080 con 0x7ff344108790 2024-04-25T01:35:52.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.910+0000 7ff349f94700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff344108790 0x7ff34419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:52.092 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.910+0000 7ff349f94700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff344108790 0x7ff34419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33612/0 (socket says 172.21.15.78:33612) 2024-04-25T01:35:52.093 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.910+0000 7ff349f94700 1 -- 172.21.15.78:0/1484636842 learned_addr learned my addr 172.21.15.78:0/1484636842 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:52.093 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.910+0000 7ff349f94700 1 -- 172.21.15.78:0/1484636842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34419c120 con 0x7ff344108790 2024-04-25T01:35:52.093 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.911+0000 7ff349f94700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff344108790 0x7ff34419bbe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff334011b60 tx=0x7ff334011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:52.093 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.911+0000 7ff33affd700 1 -- 172.21.15.78:0/1484636842 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff334003b20 con 0x7ff344108790 2024-04-25T01:35:52.093 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.911+0000 7ff33affd700 1 -- 172.21.15.78:0/1484636842 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff334003cc0 con 0x7ff344108790 2024-04-25T01:35:52.093 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.911+0000 7ff34c1f8700 1 -- 172.21.15.78:0/1484636842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34419c3b0 con 0x7ff344108790 2024-04-25T01:35:52.093 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.911+0000 7ff33affd700 1 -- 172.21.15.78:0/1484636842 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff334024840 con 0x7ff344108790 2024-04-25T01:35:52.093 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.911+0000 7ff34c1f8700 1 -- 172.21.15.78:0/1484636842 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34419f0a0 con 0x7ff344108790 2024-04-25T01:35:52.093 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.912+0000 7ff33affd700 1 -- 172.21.15.78:0/1484636842 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7ff334022020 con 0x7ff344108790 2024-04-25T01:35:52.093 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.912+0000 7ff33affd700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7ff3300390a0 0x7ff33003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:52.093 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.912+0000 7ff34c1f8700 1 -- 172.21.15.78:0/1484636842 --> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff344063890 con 0x7ff3300390a0 2024-04-25T01:35:52.093 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.912+0000 7ff33affd700 1 -- 172.21.15.78:0/1484636842 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff334011e00 con 0x7ff344108790 2024-04-25T01:35:52.094 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.912+0000 7ff349793700 1 -- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7ff3300390a0 msgr2=0x7ff33003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1544860340 2024-04-25T01:35:52.094 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:45.912+0000 7ff349793700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7ff3300390a0 0x7ff33003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:35:52.094 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:46.113+0000 7ff349793700 1 -- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7ff3300390a0 msgr2=0x7ff33003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1544860340 2024-04-25T01:35:52.094 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:46.113+0000 7ff349793700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7ff3300390a0 0x7ff33003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:35:52.094 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:46.514+0000 7ff349793700 1 -- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7ff3300390a0 msgr2=0x7ff33003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1544860340 2024-04-25T01:35:52.094 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:46.514+0000 7ff349793700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7ff3300390a0 0x7ff33003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:35:52.094 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:47.315+0000 7ff349793700 1 -- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7ff3300390a0 msgr2=0x7ff33003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1544860340 2024-04-25T01:35:52.094 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:47.315+0000 7ff349793700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7ff3300390a0 0x7ff33003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:35:52.094 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:48.916+0000 7ff349793700 1 -- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7ff3300390a0 msgr2=0x7ff33003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1544860340 2024-04-25T01:35:52.094 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:48.916+0000 7ff349793700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7ff3300390a0 0x7ff33003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T01:35:52.094 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:50.508+0000 7ff33affd700 1 -- 172.21.15.78:0/1484636842 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7ff33402d910 con 0x7ff344108790 2024-04-25T01:35:52.094 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:50.508+0000 7ff33affd700 1 -- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7ff3300390a0 msgr2=0x7ff33003b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:35:52.094 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:50.508+0000 7ff33affd700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7ff3300390a0 0x7ff33003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:52.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.510+0000 7ff33affd700 1 -- 172.21.15.78:0/1484636842 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7ff33404ea80 con 0x7ff344108790 2024-04-25T01:35:52.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.510+0000 7ff33affd700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff33003c7e0 0x7ff33003ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:52.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.511+0000 7ff33affd700 1 -- 172.21.15.78:0/1484636842 --> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff344063890 con 0x7ff33003c7e0 2024-04-25T01:35:52.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.511+0000 7ff349793700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff33003c7e0 0x7ff33003ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:52.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.511+0000 7ff349793700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff33003c7e0 0x7ff33003ebd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff340003a10 tx=0x7ff3400092b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:52.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.512+0000 7ff33affd700 1 -- 172.21.15.78:0/1484636842 <== mgr.14116 v2:172.21.15.78:6800/1101495738 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ff344063890 con 0x7ff33003c7e0 2024-04-25T01:35:52.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.518+0000 7ff34c1f8700 1 -- 172.21.15.78:0/1484636842 --> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff344103320 con 0x7ff33003c7e0 2024-04-25T01:35:52.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.518+0000 7ff33affd700 1 -- 172.21.15.78:0/1484636842 <== mgr.14116 v2:172.21.15.78:6800/1101495738 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff344103320 con 0x7ff33003c7e0 2024-04-25T01:35:52.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.519+0000 7ff34c1f8700 1 -- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff33003c7e0 msgr2=0x7ff33003ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:52.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.519+0000 7ff34c1f8700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff33003c7e0 0x7ff33003ebd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff340003a10 tx=0x7ff3400092b0 comp rx=0 tx=0).stop 2024-04-25T01:35:52.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.519+0000 7ff34c1f8700 1 -- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff344108790 msgr2=0x7ff34419bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:52.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.519+0000 7ff34c1f8700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff344108790 0x7ff34419bbe0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff334011b60 tx=0x7ff334011c40 comp rx=0 tx=0).stop 2024-04-25T01:35:52.095 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.519+0000 7ff34c1f8700 1 -- 172.21.15.78:0/1484636842 shutdown_connections 2024-04-25T01:35:52.096 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.519+0000 7ff34c1f8700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff33003c7e0 0x7ff33003ebd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:52.096 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.519+0000 7ff34c1f8700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff344108790 0x7ff34419bbe0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:52.096 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.519+0000 7ff34c1f8700 1 --2- 172.21.15.78:0/1484636842 >> [v2:172.21.15.78:6800/1544860340,v1:172.21.15.78:6801/1544860340] conn(0x7ff3300390a0 0x7ff33003b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:52.096 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.519+0000 7ff34c1f8700 1 -- 172.21.15.78:0/1484636842 >> 172.21.15.78:0/1484636842 conn(0x7ff3441021a0 msgr2=0x7ff344102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:52.096 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.519+0000 7ff34c1f8700 1 -- 172.21.15.78:0/1484636842 shutdown_connections 2024-04-25T01:35:52.096 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:51.519+0000 7ff34c1f8700 1 -- 172.21.15.78:0/1484636842 wait complete. 2024-04-25T01:35:52.096 INFO:teuthology.orchestra.run.smithi078.stdout:mgr epoch 5 is available 2024-04-25T01:35:52.096 INFO:teuthology.orchestra.run.smithi078.stdout:Setting orchestrator backend to cephadm... 2024-04-25T01:35:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:52 smithi078 ceph-mon[29469]: [25/Apr/2024:01:35:51] ENGINE Bus STARTING 2024-04-25T01:35:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:52 smithi078 ceph-mon[29469]: [25/Apr/2024:01:35:51] ENGINE Serving on http://172.21.15.78:8765 2024-04-25T01:35:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:52 smithi078 ceph-mon[29469]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T01:35:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:52 smithi078 ceph-mon[29469]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T01:35:52.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:52 smithi078 ceph-mon[29469]: [25/Apr/2024:01:35:51] ENGINE Serving on https://172.21.15.78:7150 2024-04-25T01:35:52.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:52 smithi078 ceph-mon[29469]: [25/Apr/2024:01:35:51] ENGINE Bus STARTED 2024-04-25T01:35:52.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:52 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:35:53.276 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.563+0000 7f8a8ef89700 1 Processor -- start 2024-04-25T01:35:53.276 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.563+0000 7f8a8ef89700 1 -- start start 2024-04-25T01:35:53.276 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.563+0000 7f8a8ef89700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a88106be0 0x7f8a88107000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:53.276 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.564+0000 7f8a8ef89700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a881075d0 con 0x7f8a88106be0 2024-04-25T01:35:53.276 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.564+0000 7f8a8cd25700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a88106be0 0x7f8a88107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:53.276 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.564+0000 7f8a8cd25700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a88106be0 0x7f8a88107000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51338/0 (socket says 172.21.15.78:51338) 2024-04-25T01:35:53.277 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.564+0000 7f8a8cd25700 1 -- 172.21.15.78:0/2112705550 learned_addr learned my addr 172.21.15.78:0/2112705550 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:53.277 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.564+0000 7f8a8cd25700 1 -- 172.21.15.78:0/2112705550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a88107d90 con 0x7f8a88106be0 2024-04-25T01:35:53.277 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.564+0000 7f8a8cd25700 1 --2- 172.21.15.78:0/2112705550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a88106be0 0x7f8a88107000 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0098a0 tx=0x7f8a7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330a7ffa2086a3eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:53.277 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.565+0000 7f8a877fe700 1 -- 172.21.15.78:0/2112705550 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a7c004160 con 0x7f8a88106be0 2024-04-25T01:35:53.277 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.565+0000 7f8a877fe700 1 -- 172.21.15.78:0/2112705550 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8a7c004300 con 0x7f8a88106be0 2024-04-25T01:35:53.277 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.565+0000 7f8a877fe700 1 -- 172.21.15.78:0/2112705550 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a7c011b50 con 0x7f8a88106be0 2024-04-25T01:35:53.277 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.566+0000 7f8a8ef89700 1 -- 172.21.15.78:0/2112705550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a88106be0 msgr2=0x7f8a88107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:53.277 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.566+0000 7f8a8ef89700 1 --2- 172.21.15.78:0/2112705550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a88106be0 0x7f8a88107000 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0098a0 tx=0x7f8a7c009bb0 comp rx=0 tx=0).stop 2024-04-25T01:35:53.277 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.566+0000 7f8a8ef89700 1 -- 172.21.15.78:0/2112705550 shutdown_connections 2024-04-25T01:35:53.277 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.566+0000 7f8a8ef89700 1 --2- 172.21.15.78:0/2112705550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a88106be0 0x7f8a88107000 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:53.277 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.566+0000 7f8a8ef89700 1 -- 172.21.15.78:0/2112705550 >> 172.21.15.78:0/2112705550 conn(0x7f8a881021a0 msgr2=0x7f8a881045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:53.277 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.566+0000 7f8a8ef89700 1 -- 172.21.15.78:0/2112705550 shutdown_connections 2024-04-25T01:35:53.277 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.566+0000 7f8a8ef89700 1 -- 172.21.15.78:0/2112705550 wait complete. 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.566+0000 7f8a8ef89700 1 Processor -- start 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.566+0000 7f8a8ef89700 1 -- start start 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.567+0000 7f8a8ef89700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a88106be0 0x7f8a8819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.567+0000 7f8a8ef89700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8810b140 con 0x7f8a88106be0 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.567+0000 7f8a8cd25700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a88106be0 0x7f8a8819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.567+0000 7f8a8cd25700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a88106be0 0x7f8a8819bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51342/0 (socket says 172.21.15.78:51342) 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.567+0000 7f8a8cd25700 1 -- 172.21.15.78:0/24019987 learned_addr learned my addr 172.21.15.78:0/24019987 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.567+0000 7f8a8cd25700 1 -- 172.21.15.78:0/24019987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8819c190 con 0x7f8a88106be0 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.567+0000 7f8a8cd25700 1 --2- 172.21.15.78:0/24019987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a88106be0 0x7f8a8819bc50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c011670 tx=0x7f8a7c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.567+0000 7f8a85ffb700 1 -- 172.21.15.78:0/24019987 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a7c0039c0 con 0x7f8a88106be0 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.567+0000 7f8a85ffb700 1 -- 172.21.15.78:0/24019987 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8a7c011e50 con 0x7f8a88106be0 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.567+0000 7f8a8ef89700 1 -- 172.21.15.78:0/24019987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8819c420 con 0x7f8a88106be0 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.567+0000 7f8a85ffb700 1 -- 172.21.15.78:0/24019987 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a7c01a5c0 con 0x7f8a88106be0 2024-04-25T01:35:53.278 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.568+0000 7f8a8ef89700 1 -- 172.21.15.78:0/24019987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a8819f110 con 0x7f8a88106be0 2024-04-25T01:35:53.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.568+0000 7f8a85ffb700 1 -- 172.21.15.78:0/24019987 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8a7c01a870 con 0x7f8a88106be0 2024-04-25T01:35:53.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.568+0000 7f8a8ef89700 1 -- 172.21.15.78:0/24019987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a88063890 con 0x7f8a88106be0 2024-04-25T01:35:53.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.568+0000 7f8a85ffb700 1 --2- 172.21.15.78:0/24019987 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f8a78039050 0x7f8a7803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:53.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.569+0000 7f8a85ffb700 1 -- 172.21.15.78:0/24019987 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8a7c04d460 con 0x7f8a88106be0 2024-04-25T01:35:53.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.569+0000 7f8a87fff700 1 --2- 172.21.15.78:0/24019987 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f8a78039050 0x7f8a7803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:53.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.569+0000 7f8a87fff700 1 --2- 172.21.15.78:0/24019987 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f8a78039050 0x7f8a7803b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8a74006fd0 tx=0x7f8a74006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:53.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.574+0000 7f8a85ffb700 1 -- 172.21.15.78:0/24019987 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8a7c019c70 con 0x7f8a88106be0 2024-04-25T01:35:53.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.788+0000 7f8a8ef89700 1 -- 172.21.15.78:0/24019987 --> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f8a880008d0 con 0x7f8a78039050 2024-04-25T01:35:53.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.794+0000 7f8a85ffb700 1 -- 172.21.15.78:0/24019987 <== mgr.14116 v2:172.21.15.78:6800/1101495738 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8a880008d0 con 0x7f8a78039050 2024-04-25T01:35:53.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.796+0000 7f8a8ef89700 1 -- 172.21.15.78:0/24019987 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f8a78039050 msgr2=0x7f8a7803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:53.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.796+0000 7f8a8ef89700 1 --2- 172.21.15.78:0/24019987 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f8a78039050 0x7f8a7803b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8a74006fd0 tx=0x7f8a74006e40 comp rx=0 tx=0).stop 2024-04-25T01:35:53.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.796+0000 7f8a8ef89700 1 -- 172.21.15.78:0/24019987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a88106be0 msgr2=0x7f8a8819bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:53.279 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.796+0000 7f8a8ef89700 1 --2- 172.21.15.78:0/24019987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a88106be0 0x7f8a8819bc50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c011670 tx=0x7f8a7c011b60 comp rx=0 tx=0).stop 2024-04-25T01:35:53.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.796+0000 7f8a8ef89700 1 -- 172.21.15.78:0/24019987 shutdown_connections 2024-04-25T01:35:53.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.796+0000 7f8a8ef89700 1 --2- 172.21.15.78:0/24019987 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f8a78039050 0x7f8a7803b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:53.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.796+0000 7f8a8ef89700 1 --2- 172.21.15.78:0/24019987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8a88106be0 0x7f8a8819bc50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:53.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.796+0000 7f8a8ef89700 1 -- 172.21.15.78:0/24019987 >> 172.21.15.78:0/24019987 conn(0x7f8a881021a0 msgr2=0x7f8a88103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:53.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.796+0000 7f8a8ef89700 1 -- 172.21.15.78:0/24019987 shutdown_connections 2024-04-25T01:35:53.280 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:52.796+0000 7f8a8ef89700 1 -- 172.21.15.78:0/24019987 wait complete. 2024-04-25T01:35:53.663 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:53 smithi078 ceph-mon[29469]: mgrmap e8: x(active, since 2s) 2024-04-25T01:35:53.663 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:53 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' 2024-04-25T01:35:53.663 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:53 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:35:54.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T01:35:54.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.712+0000 7fc18a5f2700 1 Processor -- start 2024-04-25T01:35:54.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.712+0000 7fc18a5f2700 1 -- start start 2024-04-25T01:35:54.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.712+0000 7fc18a5f2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc184108840 0x7fc18410ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:54.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.712+0000 7fc18a5f2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc18407c660 con 0x7fc184108840 2024-04-25T01:35:54.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.712+0000 7fc183fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc184108840 0x7fc18410ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:54.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.712+0000 7fc183fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc184108840 0x7fc18410ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51358/0 (socket says 172.21.15.78:51358) 2024-04-25T01:35:54.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.712+0000 7fc183fff700 1 -- 172.21.15.78:0/958284711 learned_addr learned my addr 172.21.15.78:0/958284711 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:54.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.712+0000 7fc183fff700 1 -- 172.21.15.78:0/958284711 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18410b170 con 0x7fc184108840 2024-04-25T01:35:54.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.713+0000 7fc183fff700 1 --2- 172.21.15.78:0/958284711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc184108840 0x7fc18410ac30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc1740098a0 tx=0x7fc174009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552ab064769bc882 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:54.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.713+0000 7fc182ffd700 1 -- 172.21.15.78:0/958284711 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc174004160 con 0x7fc184108840 2024-04-25T01:35:54.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.713+0000 7fc182ffd700 1 -- 172.21.15.78:0/958284711 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc174004300 con 0x7fc184108840 2024-04-25T01:35:54.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.713+0000 7fc182ffd700 1 -- 172.21.15.78:0/958284711 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc174011b50 con 0x7fc184108840 2024-04-25T01:35:54.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.713+0000 7fc18a5f2700 1 -- 172.21.15.78:0/958284711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc184108840 msgr2=0x7fc18410ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:54.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.713+0000 7fc18a5f2700 1 --2- 172.21.15.78:0/958284711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc184108840 0x7fc18410ac30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc1740098a0 tx=0x7fc174009bb0 comp rx=0 tx=0).stop 2024-04-25T01:35:54.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.713+0000 7fc18a5f2700 1 -- 172.21.15.78:0/958284711 shutdown_connections 2024-04-25T01:35:54.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.713+0000 7fc18a5f2700 1 --2- 172.21.15.78:0/958284711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc184108840 0x7fc18410ac30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:54.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.713+0000 7fc18a5f2700 1 -- 172.21.15.78:0/958284711 >> 172.21.15.78:0/958284711 conn(0x7fc184102250 msgr2=0x7fc184104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:54.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.713+0000 7fc18a5f2700 1 -- 172.21.15.78:0/958284711 shutdown_connections 2024-04-25T01:35:54.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.713+0000 7fc18a5f2700 1 -- 172.21.15.78:0/958284711 wait complete. 2024-04-25T01:35:54.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.714+0000 7fc18a5f2700 1 Processor -- start 2024-04-25T01:35:54.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.714+0000 7fc18a5f2700 1 -- start start 2024-04-25T01:35:54.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.714+0000 7fc18a5f2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc184108840 0x7fc18419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:54.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.714+0000 7fc18a5f2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc18410c130 con 0x7fc184108840 2024-04-25T01:35:54.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.714+0000 7fc183fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc184108840 0x7fc18419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:54.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.715+0000 7fc183fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc184108840 0x7fc18419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51370/0 (socket says 172.21.15.78:51370) 2024-04-25T01:35:54.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.715+0000 7fc183fff700 1 -- 172.21.15.78:0/1149950134 learned_addr learned my addr 172.21.15.78:0/1149950134 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:54.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.715+0000 7fc183fff700 1 -- 172.21.15.78:0/1149950134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18419c1b0 con 0x7fc184108840 2024-04-25T01:35:54.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.715+0000 7fc183fff700 1 --2- 172.21.15.78:0/1149950134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc184108840 0x7fc18419bc70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc174011670 tx=0x7fc174011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:54.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.715+0000 7fc1817fa700 1 -- 172.21.15.78:0/1149950134 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1740039c0 con 0x7fc184108840 2024-04-25T01:35:54.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.715+0000 7fc18a5f2700 1 -- 172.21.15.78:0/1149950134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18419c440 con 0x7fc184108840 2024-04-25T01:35:54.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.715+0000 7fc1817fa700 1 -- 172.21.15.78:0/1149950134 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc174011e50 con 0x7fc184108840 2024-04-25T01:35:54.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.715+0000 7fc1817fa700 1 -- 172.21.15.78:0/1149950134 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc17401a5c0 con 0x7fc184108840 2024-04-25T01:35:54.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.715+0000 7fc18a5f2700 1 -- 172.21.15.78:0/1149950134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc18419f130 con 0x7fc184108840 2024-04-25T01:35:54.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.716+0000 7fc1817fa700 1 -- 172.21.15.78:0/1149950134 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc17401a870 con 0x7fc184108840 2024-04-25T01:35:54.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.716+0000 7fc1817fa700 1 --2- 172.21.15.78:0/1149950134 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fc16c039050 0x7fc16c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:54.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.716+0000 7fc1817fa700 1 -- 172.21.15.78:0/1149950134 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc17404d360 con 0x7fc184108840 2024-04-25T01:35:54.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.716+0000 7fc18a5f2700 1 -- 172.21.15.78:0/1149950134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc184063890 con 0x7fc184108840 2024-04-25T01:35:54.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.716+0000 7fc1837fe700 1 --2- 172.21.15.78:0/1149950134 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fc16c039050 0x7fc16c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:54.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.717+0000 7fc1837fe700 1 --2- 172.21.15.78:0/1149950134 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fc16c039050 0x7fc16c03b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc178006fd0 tx=0x7fc178006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:54.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.721+0000 7fc1817fa700 1 -- 172.21.15.78:0/1149950134 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc174019c70 con 0x7fc184108840 2024-04-25T01:35:54.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.917+0000 7fc18a5f2700 1 -- 172.21.15.78:0/1149950134 --> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fc1841033b0 con 0x7fc16c039050 2024-04-25T01:35:54.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.917+0000 7fc1817fa700 1 -- 172.21.15.78:0/1149950134 <== mgr.14116 v2:172.21.15.78:6800/1101495738 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fc1841033b0 con 0x7fc16c039050 2024-04-25T01:35:54.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.919+0000 7fc18a5f2700 1 -- 172.21.15.78:0/1149950134 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fc16c039050 msgr2=0x7fc16c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:54.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.919+0000 7fc18a5f2700 1 --2- 172.21.15.78:0/1149950134 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fc16c039050 0x7fc16c03b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc178006fd0 tx=0x7fc178006e40 comp rx=0 tx=0).stop 2024-04-25T01:35:54.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.919+0000 7fc18a5f2700 1 -- 172.21.15.78:0/1149950134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc184108840 msgr2=0x7fc18419bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:54.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.919+0000 7fc18a5f2700 1 --2- 172.21.15.78:0/1149950134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc184108840 0x7fc18419bc70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc174011670 tx=0x7fc174011b60 comp rx=0 tx=0).stop 2024-04-25T01:35:54.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.919+0000 7fc18a5f2700 1 -- 172.21.15.78:0/1149950134 shutdown_connections 2024-04-25T01:35:54.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.919+0000 7fc18a5f2700 1 --2- 172.21.15.78:0/1149950134 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fc16c039050 0x7fc16c03b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:54.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.919+0000 7fc18a5f2700 1 --2- 172.21.15.78:0/1149950134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc184108840 0x7fc18419bc70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:54.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.919+0000 7fc18a5f2700 1 -- 172.21.15.78:0/1149950134 >> 172.21.15.78:0/1149950134 conn(0x7fc184102250 msgr2=0x7fc184102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:54.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.919+0000 7fc18a5f2700 1 -- 172.21.15.78:0/1149950134 shutdown_connections 2024-04-25T01:35:54.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:53.919+0000 7fc18a5f2700 1 -- 172.21.15.78:0/1149950134 wait complete. 2024-04-25T01:35:54.503 INFO:teuthology.orchestra.run.smithi078.stdout:Generating ssh key... 2024-04-25T01:35:54.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:54 smithi078 ceph-mon[29469]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:55.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-mon[29469]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:55.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' 2024-04-25T01:35:55.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' 2024-04-25T01:35:55.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: Generating public/private rsa key pair. 2024-04-25T01:35:55.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: Your identification has been saved in /tmp/tmp5jhq9s2_/key. 2024-04-25T01:35:55.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: Your public key has been saved in /tmp/tmp5jhq9s2_/key.pub. 2024-04-25T01:35:55.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: The key fingerprint is: 2024-04-25T01:35:55.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: SHA256:irqhJPxnFGCcnlWJcjWnlfFs5qgJhnO6mkhlNPjA3nk ceph-df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:35:55.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: The key's randomart image is: 2024-04-25T01:35:55.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: +---[RSA 3072]----+ 2024-04-25T01:35:55.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: | . . ++.+o | 2024-04-25T01:35:55.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: |. .* + .=.o | 2024-04-25T01:35:55.699 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: | +oo* . = | 2024-04-25T01:35:55.699 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: |. =o+. = | 2024-04-25T01:35:55.699 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: | . O E. S. . | 2024-04-25T01:35:55.699 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: |. o *o..o | 2024-04-25T01:35:55.699 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: |.+..o .o | 2024-04-25T01:35:55.699 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: |=.oo.o | 2024-04-25T01:35:55.699 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: |oo++o | 2024-04-25T01:35:55.699 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:35:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: +----[SHA256]-----+ 2024-04-25T01:35:55.980 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.956+0000 7f31abc27700 1 Processor -- start 2024-04-25T01:35:55.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.956+0000 7f31abc27700 1 -- start start 2024-04-25T01:35:55.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.956+0000 7f31abc27700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4106c30 0x7f31a4107050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:55.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.956+0000 7f31abc27700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31a4107620 con 0x7f31a4106c30 2024-04-25T01:35:55.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.956+0000 7f31a99c3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4106c30 0x7f31a4107050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:55.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.956+0000 7f31a99c3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4106c30 0x7f31a4107050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51382/0 (socket says 172.21.15.78:51382) 2024-04-25T01:35:55.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.956+0000 7f31a99c3700 1 -- 172.21.15.78:0/2762983859 learned_addr learned my addr 172.21.15.78:0/2762983859 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:55.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.957+0000 7f31a99c3700 1 -- 172.21.15.78:0/2762983859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a4107e30 con 0x7f31a4106c30 2024-04-25T01:35:55.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.957+0000 7f31a99c3700 1 --2- 172.21.15.78:0/2762983859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4106c30 0x7f31a4107050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f31a00098a0 tx=0x7f31a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0eeb95d2e04908 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:55.981 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.957+0000 7f31a89c1700 1 -- 172.21.15.78:0/2762983859 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31a0004160 con 0x7f31a4106c30 2024-04-25T01:35:55.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.957+0000 7f31a89c1700 1 -- 172.21.15.78:0/2762983859 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f31a0004300 con 0x7f31a4106c30 2024-04-25T01:35:55.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.957+0000 7f31a89c1700 1 -- 172.21.15.78:0/2762983859 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31a0011b50 con 0x7f31a4106c30 2024-04-25T01:35:55.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.957+0000 7f31abc27700 1 -- 172.21.15.78:0/2762983859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4106c30 msgr2=0x7f31a4107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:55.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.957+0000 7f31abc27700 1 --2- 172.21.15.78:0/2762983859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4106c30 0x7f31a4107050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f31a00098a0 tx=0x7f31a0009bb0 comp rx=0 tx=0).stop 2024-04-25T01:35:55.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.957+0000 7f31abc27700 1 -- 172.21.15.78:0/2762983859 shutdown_connections 2024-04-25T01:35:55.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.957+0000 7f31abc27700 1 --2- 172.21.15.78:0/2762983859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4106c30 0x7f31a4107050 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:55.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.957+0000 7f31abc27700 1 -- 172.21.15.78:0/2762983859 >> 172.21.15.78:0/2762983859 conn(0x7f31a4102250 msgr2=0x7f31a4104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:55.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.958+0000 7f31abc27700 1 -- 172.21.15.78:0/2762983859 shutdown_connections 2024-04-25T01:35:55.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.958+0000 7f31abc27700 1 -- 172.21.15.78:0/2762983859 wait complete. 2024-04-25T01:35:55.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.958+0000 7f31abc27700 1 Processor -- start 2024-04-25T01:35:55.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.958+0000 7f31abc27700 1 -- start start 2024-04-25T01:35:55.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.958+0000 7f31abc27700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4106c30 0x7f31a407c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:55.982 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.958+0000 7f31abc27700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31a410b1e0 con 0x7f31a4106c30 2024-04-25T01:35:55.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.958+0000 7f31a99c3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4106c30 0x7f31a407c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:55.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.958+0000 7f31a99c3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4106c30 0x7f31a407c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51390/0 (socket says 172.21.15.78:51390) 2024-04-25T01:35:55.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.958+0000 7f31a99c3700 1 -- 172.21.15.78:0/3323887236 learned_addr learned my addr 172.21.15.78:0/3323887236 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:55.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.959+0000 7f31a99c3700 1 -- 172.21.15.78:0/3323887236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a419e9f0 con 0x7f31a4106c30 2024-04-25T01:35:55.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.959+0000 7f31a99c3700 1 --2- 172.21.15.78:0/3323887236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4106c30 0x7f31a407c2a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f31a0011670 tx=0x7f31a0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:55.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.959+0000 7f319affd700 1 -- 172.21.15.78:0/3323887236 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31a00039c0 con 0x7f31a4106c30 2024-04-25T01:35:55.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.959+0000 7f319affd700 1 -- 172.21.15.78:0/3323887236 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f31a0011e50 con 0x7f31a4106c30 2024-04-25T01:35:55.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.959+0000 7f319affd700 1 -- 172.21.15.78:0/3323887236 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31a001a5c0 con 0x7f31a4106c30 2024-04-25T01:35:55.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.959+0000 7f31abc27700 1 -- 172.21.15.78:0/3323887236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a419ebc0 con 0x7f31a4106c30 2024-04-25T01:35:55.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.959+0000 7f31abc27700 1 -- 172.21.15.78:0/3323887236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a419ef60 con 0x7f31a4106c30 2024-04-25T01:35:55.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.960+0000 7f319affd700 1 -- 172.21.15.78:0/3323887236 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f31a001a870 con 0x7f31a4106c30 2024-04-25T01:35:55.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.960+0000 7f31abc27700 1 -- 172.21.15.78:0/3323887236 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a4063890 con 0x7f31a4106c30 2024-04-25T01:35:55.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.960+0000 7f319affd700 1 --2- 172.21.15.78:0/3323887236 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f3190039050 0x7f319003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:55.983 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.960+0000 7f319affd700 1 -- 172.21.15.78:0/3323887236 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f31a004d420 con 0x7f31a4106c30 2024-04-25T01:35:55.984 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.960+0000 7f31a91c2700 1 --2- 172.21.15.78:0/3323887236 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f3190039050 0x7f319003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:55.984 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.960+0000 7f31a91c2700 1 --2- 172.21.15.78:0/3323887236 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f3190039050 0x7f319003b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3194006fd0 tx=0x7f3194006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:55.984 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:54.966+0000 7f319affd700 1 -- 172.21.15.78:0/3323887236 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f31a000f2e0 con 0x7f31a4106c30 2024-04-25T01:35:55.984 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:55.158+0000 7f31abc27700 1 -- 172.21.15.78:0/3323887236 --> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f31a41033d0 con 0x7f3190039050 2024-04-25T01:35:55.984 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:55.423+0000 7f319affd700 1 -- 172.21.15.78:0/3323887236 <== mgr.14116 v2:172.21.15.78:6800/1101495738 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f31a41033d0 con 0x7f3190039050 2024-04-25T01:35:55.984 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:55.426+0000 7f31abc27700 1 -- 172.21.15.78:0/3323887236 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f3190039050 msgr2=0x7f319003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:55.984 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:55.426+0000 7f31abc27700 1 --2- 172.21.15.78:0/3323887236 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f3190039050 0x7f319003b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3194006fd0 tx=0x7f3194006e40 comp rx=0 tx=0).stop 2024-04-25T01:35:55.984 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:55.426+0000 7f31abc27700 1 -- 172.21.15.78:0/3323887236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4106c30 msgr2=0x7f31a407c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:55.984 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:55.426+0000 7f31abc27700 1 --2- 172.21.15.78:0/3323887236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4106c30 0x7f31a407c2a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f31a0011670 tx=0x7f31a0011b60 comp rx=0 tx=0).stop 2024-04-25T01:35:55.984 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:55.426+0000 7f31abc27700 1 -- 172.21.15.78:0/3323887236 shutdown_connections 2024-04-25T01:35:55.984 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:55.426+0000 7f31abc27700 1 --2- 172.21.15.78:0/3323887236 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f3190039050 0x7f319003b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:55.984 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:55.426+0000 7f31abc27700 1 --2- 172.21.15.78:0/3323887236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a4106c30 0x7f31a407c2a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:55.985 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:55.426+0000 7f31abc27700 1 -- 172.21.15.78:0/3323887236 >> 172.21.15.78:0/3323887236 conn(0x7f31a4102250 msgr2=0x7f31a4102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:55.985 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:55.426+0000 7f31abc27700 1 -- 172.21.15.78:0/3323887236 shutdown_connections 2024-04-25T01:35:55.985 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:55.426+0000 7f31abc27700 1 -- 172.21.15.78:0/3323887236 wait complete. 2024-04-25T01:35:56.611 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:56 smithi078 ceph-mon[29469]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:56.611 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:56 smithi078 ceph-mon[29469]: Generating ssh key... 2024-04-25T01:35:57.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFGDsYAnYi1SICvR6km81JBw2ZhEj7TRQYaAnyOMmj08Skwc6Pd22INWVW6AWoPorTbcxYNkCym1Vt1CGKVRupd4T6VO4C1XO+8T/xjepKXR/Q9ams33Dpq8A3cMyLmef66eRsvSpaKoWwO9e3BawuKsfj1Mzdolnh1+kgChZjNxNmAHCwLjRBVeXWHZhl4uxDcuEI/lgBAa+1WVfcUhkkqoUeF6IHJx0HRdDYOImFO8qB+twCGoS01bJ+aQEoMPOqv6+qwS4f4mZGBQ47aBQpc28IV3ujGCQZdLBRRSXV6Ric2bwbcqIZH7Tmida9j8emSPP8mpOL1Y+uvaD25n3nXc6uwl4ZUUtfRi3GMb8vIp0oFLkCBxQwvC/lJ+VN1pLFodDzzJIHZ1jfF/KCAsewQYHRa5iy5DXDxX7WJ9KHcETu654640/krxwxR7WfLukK5kP14sRbkoPQ+6el7mTQSJ6rLyRN+iUMMhNrEUJxx7wf3p0S75seauZPL5g90wE= ceph-df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:35:57.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.428+0000 7ff143745700 1 Processor -- start 2024-04-25T01:35:57.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.428+0000 7ff143745700 1 -- start start 2024-04-25T01:35:57.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.428+0000 7ff143745700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c107d60 0x7ff13c108180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:57.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.428+0000 7ff143745700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c108750 con 0x7ff13c107d60 2024-04-25T01:35:57.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.428+0000 7ff1414e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c107d60 0x7ff13c108180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:57.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.428+0000 7ff1414e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c107d60 0x7ff13c108180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51394/0 (socket says 172.21.15.78:51394) 2024-04-25T01:35:57.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.428+0000 7ff1414e1700 1 -- 172.21.15.78:0/237897891 learned_addr learned my addr 172.21.15.78:0/237897891 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:57.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.429+0000 7ff1414e1700 1 -- 172.21.15.78:0/237897891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c108ec0 con 0x7ff13c107d60 2024-04-25T01:35:57.162 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.429+0000 7ff1414e1700 1 --2- 172.21.15.78:0/237897891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c107d60 0x7ff13c108180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff1380098a0 tx=0x7ff138009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ac6d7d75df981b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:57.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.429+0000 7ff133fff700 1 -- 172.21.15.78:0/237897891 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff138004160 con 0x7ff13c107d60 2024-04-25T01:35:57.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.429+0000 7ff133fff700 1 -- 172.21.15.78:0/237897891 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff138004300 con 0x7ff13c107d60 2024-04-25T01:35:57.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.429+0000 7ff133fff700 1 -- 172.21.15.78:0/237897891 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff138011b50 con 0x7ff13c107d60 2024-04-25T01:35:57.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.429+0000 7ff143745700 1 -- 172.21.15.78:0/237897891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c107d60 msgr2=0x7ff13c108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:57.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.429+0000 7ff143745700 1 --2- 172.21.15.78:0/237897891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c107d60 0x7ff13c108180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff1380098a0 tx=0x7ff138009bb0 comp rx=0 tx=0).stop 2024-04-25T01:35:57.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.430+0000 7ff143745700 1 -- 172.21.15.78:0/237897891 shutdown_connections 2024-04-25T01:35:57.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.430+0000 7ff143745700 1 --2- 172.21.15.78:0/237897891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c107d60 0x7ff13c108180 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:57.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.430+0000 7ff143745700 1 -- 172.21.15.78:0/237897891 >> 172.21.15.78:0/237897891 conn(0x7ff13c103300 msgr2=0x7ff13c105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:57.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.430+0000 7ff143745700 1 -- 172.21.15.78:0/237897891 shutdown_connections 2024-04-25T01:35:57.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.430+0000 7ff143745700 1 -- 172.21.15.78:0/237897891 wait complete. 2024-04-25T01:35:57.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.430+0000 7ff143745700 1 Processor -- start 2024-04-25T01:35:57.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.430+0000 7ff143745700 1 -- start start 2024-04-25T01:35:57.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.430+0000 7ff143745700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c107d60 0x7ff13c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:57.163 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.430+0000 7ff143745700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c10c270 con 0x7ff13c107d60 2024-04-25T01:35:57.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.431+0000 7ff1414e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c107d60 0x7ff13c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:57.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.431+0000 7ff1414e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c107d60 0x7ff13c19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51400/0 (socket says 172.21.15.78:51400) 2024-04-25T01:35:57.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.431+0000 7ff1414e1700 1 -- 172.21.15.78:0/48362791 learned_addr learned my addr 172.21.15.78:0/48362791 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:35:57.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.431+0000 7ff1414e1700 1 -- 172.21.15.78:0/48362791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c19c1f0 con 0x7ff13c107d60 2024-04-25T01:35:57.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.431+0000 7ff1414e1700 1 --2- 172.21.15.78:0/48362791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c107d60 0x7ff13c19bcb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff138011670 tx=0x7ff138011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:57.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.431+0000 7ff1327fc700 1 -- 172.21.15.78:0/48362791 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1380039c0 con 0x7ff13c107d60 2024-04-25T01:35:57.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.431+0000 7ff1327fc700 1 -- 172.21.15.78:0/48362791 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff138011e50 con 0x7ff13c107d60 2024-04-25T01:35:57.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.431+0000 7ff143745700 1 -- 172.21.15.78:0/48362791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13c19c480 con 0x7ff13c107d60 2024-04-25T01:35:57.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.431+0000 7ff1327fc700 1 -- 172.21.15.78:0/48362791 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff13801a5c0 con 0x7ff13c107d60 2024-04-25T01:35:57.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.431+0000 7ff143745700 1 -- 172.21.15.78:0/48362791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13c19f170 con 0x7ff13c107d60 2024-04-25T01:35:57.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.432+0000 7ff1327fc700 1 -- 172.21.15.78:0/48362791 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff13801a870 con 0x7ff13c107d60 2024-04-25T01:35:57.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.432+0000 7ff143745700 1 -- 172.21.15.78:0/48362791 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff13c063890 con 0x7ff13c107d60 2024-04-25T01:35:57.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.432+0000 7ff1327fc700 1 --2- 172.21.15.78:0/48362791 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff128039050 0x7ff12803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:57.164 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.432+0000 7ff1327fc700 1 -- 172.21.15.78:0/48362791 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff13804d3b0 con 0x7ff13c107d60 2024-04-25T01:35:57.165 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.432+0000 7ff140ce0700 1 --2- 172.21.15.78:0/48362791 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff128039050 0x7ff12803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:57.165 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.433+0000 7ff140ce0700 1 --2- 172.21.15.78:0/48362791 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff128039050 0x7ff12803b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff12c006fd0 tx=0x7ff12c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:57.165 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.437+0000 7ff1327fc700 1 -- 172.21.15.78:0/48362791 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff138003b60 con 0x7ff13c107d60 2024-04-25T01:35:57.165 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.635+0000 7ff143745700 1 -- 172.21.15.78:0/48362791 --> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7ff13c1045c0 con 0x7ff128039050 2024-04-25T01:35:57.165 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.636+0000 7ff1327fc700 1 -- 172.21.15.78:0/48362791 <== mgr.14116 v2:172.21.15.78:6800/1101495738 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7ff13c1045c0 con 0x7ff128039050 2024-04-25T01:35:57.165 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.639+0000 7ff143745700 1 -- 172.21.15.78:0/48362791 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff128039050 msgr2=0x7ff12803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:57.165 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.639+0000 7ff143745700 1 --2- 172.21.15.78:0/48362791 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff128039050 0x7ff12803b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff12c006fd0 tx=0x7ff12c006e40 comp rx=0 tx=0).stop 2024-04-25T01:35:57.165 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.639+0000 7ff143745700 1 -- 172.21.15.78:0/48362791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c107d60 msgr2=0x7ff13c19bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:57.166 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.639+0000 7ff143745700 1 --2- 172.21.15.78:0/48362791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c107d60 0x7ff13c19bcb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff138011670 tx=0x7ff138011b60 comp rx=0 tx=0).stop 2024-04-25T01:35:57.166 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.639+0000 7ff143745700 1 -- 172.21.15.78:0/48362791 shutdown_connections 2024-04-25T01:35:57.166 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.639+0000 7ff143745700 1 --2- 172.21.15.78:0/48362791 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff128039050 0x7ff12803b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:57.166 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.639+0000 7ff143745700 1 --2- 172.21.15.78:0/48362791 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13c107d60 0x7ff13c19bcb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:57.166 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.639+0000 7ff143745700 1 -- 172.21.15.78:0/48362791 >> 172.21.15.78:0/48362791 conn(0x7ff13c103300 msgr2=0x7ff13c103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:57.166 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.640+0000 7ff143745700 1 -- 172.21.15.78:0/48362791 shutdown_connections 2024-04-25T01:35:57.166 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:56.640+0000 7ff143745700 1 -- 172.21.15.78:0/48362791 wait complete. 2024-04-25T01:35:57.166 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T01:35:57.166 INFO:teuthology.orchestra.run.smithi078.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T01:35:57.166 INFO:teuthology.orchestra.run.smithi078.stdout:Adding host smithi078... 2024-04-25T01:35:57.741 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:57 smithi078 ceph-mon[29469]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:58.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:58 smithi078 ceph-mon[29469]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi078", "addr": "172.21.15.78", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:58.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:35:58 smithi078 ceph-mon[29469]: Deploying cephadm binary to smithi078 2024-04-25T01:36:02.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Added host 'smithi078' with addr '172.21.15.78' 2024-04-25T01:36:02.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.634+0000 7fd9eea4b700 1 Processor -- start 2024-04-25T01:36:02.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.634+0000 7fd9eea4b700 1 -- start start 2024-04-25T01:36:02.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.635+0000 7fd9eea4b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8106c00 0x7fd9e8107020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:02.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.635+0000 7fd9eea4b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e81075f0 con 0x7fd9e8106c00 2024-04-25T01:36:02.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.635+0000 7fd9e7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8106c00 0x7fd9e8107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:02.428 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.635+0000 7fd9e7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8106c00 0x7fd9e8107020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51408/0 (socket says 172.21.15.78:51408) 2024-04-25T01:36:02.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.635+0000 7fd9e7fff700 1 -- 172.21.15.78:0/803606935 learned_addr learned my addr 172.21.15.78:0/803606935 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:02.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.635+0000 7fd9e7fff700 1 -- 172.21.15.78:0/803606935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e8107e00 con 0x7fd9e8106c00 2024-04-25T01:36:02.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.635+0000 7fd9e7fff700 1 --2- 172.21.15.78:0/803606935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8106c00 0x7fd9e8107020 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80098a0 tx=0x7fd9d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ebb011a53b8c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:02.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.635+0000 7fd9e6ffd700 1 -- 172.21.15.78:0/803606935 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9d8004160 con 0x7fd9e8106c00 2024-04-25T01:36:02.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.635+0000 7fd9e6ffd700 1 -- 172.21.15.78:0/803606935 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd9d8004300 con 0x7fd9e8106c00 2024-04-25T01:36:02.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.635+0000 7fd9e6ffd700 1 -- 172.21.15.78:0/803606935 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9d8011b50 con 0x7fd9e8106c00 2024-04-25T01:36:02.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.636+0000 7fd9eea4b700 1 -- 172.21.15.78:0/803606935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8106c00 msgr2=0x7fd9e8107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:02.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.636+0000 7fd9eea4b700 1 --2- 172.21.15.78:0/803606935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8106c00 0x7fd9e8107020 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80098a0 tx=0x7fd9d8009bb0 comp rx=0 tx=0).stop 2024-04-25T01:36:02.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.636+0000 7fd9eea4b700 1 -- 172.21.15.78:0/803606935 shutdown_connections 2024-04-25T01:36:02.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.636+0000 7fd9eea4b700 1 --2- 172.21.15.78:0/803606935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8106c00 0x7fd9e8107020 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:02.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.636+0000 7fd9eea4b700 1 -- 172.21.15.78:0/803606935 >> 172.21.15.78:0/803606935 conn(0x7fd9e81021a0 msgr2=0x7fd9e81045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:02.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.636+0000 7fd9eea4b700 1 -- 172.21.15.78:0/803606935 shutdown_connections 2024-04-25T01:36:02.429 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.636+0000 7fd9eea4b700 1 -- 172.21.15.78:0/803606935 wait complete. 2024-04-25T01:36:02.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.637+0000 7fd9eea4b700 1 Processor -- start 2024-04-25T01:36:02.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.637+0000 7fd9eea4b700 1 -- start start 2024-04-25T01:36:02.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.637+0000 7fd9eea4b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8106c00 0x7fd9e819bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:02.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.637+0000 7fd9eea4b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e810b1b0 con 0x7fd9e8106c00 2024-04-25T01:36:02.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.637+0000 7fd9e7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8106c00 0x7fd9e819bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:02.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.637+0000 7fd9e7fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8106c00 0x7fd9e819bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51416/0 (socket says 172.21.15.78:51416) 2024-04-25T01:36:02.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.637+0000 7fd9e7fff700 1 -- 172.21.15.78:0/1133071394 learned_addr learned my addr 172.21.15.78:0/1133071394 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:02.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.637+0000 7fd9e7fff700 1 -- 172.21.15.78:0/1133071394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e819c200 con 0x7fd9e8106c00 2024-04-25T01:36:02.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.637+0000 7fd9e7fff700 1 --2- 172.21.15.78:0/1133071394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8106c00 0x7fd9e819bcc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8000c00 tx=0x7fd9d8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:02.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.637+0000 7fd9e57fa700 1 -- 172.21.15.78:0/1133071394 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9d8003b20 con 0x7fd9e8106c00 2024-04-25T01:36:02.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.638+0000 7fd9e57fa700 1 -- 172.21.15.78:0/1133071394 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd9d8003cc0 con 0x7fd9e8106c00 2024-04-25T01:36:02.430 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.638+0000 7fd9eea4b700 1 -- 172.21.15.78:0/1133071394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e819c490 con 0x7fd9e8106c00 2024-04-25T01:36:02.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.638+0000 7fd9e57fa700 1 -- 172.21.15.78:0/1133071394 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9d8024910 con 0x7fd9e8106c00 2024-04-25T01:36:02.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.638+0000 7fd9eea4b700 1 -- 172.21.15.78:0/1133071394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e819f180 con 0x7fd9e8106c00 2024-04-25T01:36:02.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.638+0000 7fd9e57fa700 1 -- 172.21.15.78:0/1133071394 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fd9d8022020 con 0x7fd9e8106c00 2024-04-25T01:36:02.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.639+0000 7fd9eea4b700 1 -- 172.21.15.78:0/1133071394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e8063890 con 0x7fd9e8106c00 2024-04-25T01:36:02.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.639+0000 7fd9e57fa700 1 --2- 172.21.15.78:0/1133071394 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fd9d4039050 0x7fd9d403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:02.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.639+0000 7fd9e57fa700 1 -- 172.21.15.78:0/1133071394 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd9d8011e00 con 0x7fd9e8106c00 2024-04-25T01:36:02.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.639+0000 7fd9e77fe700 1 --2- 172.21.15.78:0/1133071394 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fd9d4039050 0x7fd9d403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:02.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.639+0000 7fd9e77fe700 1 --2- 172.21.15.78:0/1133071394 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fd9d4039050 0x7fd9d403b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0006fd0 tx=0x7fd9d0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:02.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.644+0000 7fd9e57fa700 1 -- 172.21.15.78:0/1133071394 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd9d8017ca0 con 0x7fd9e8106c00 2024-04-25T01:36:02.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:35:57.843+0000 7fd9eea4b700 1 -- 172.21.15.78:0/1133071394 --> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi078", "addr": "172.21.15.78", "target": ["mon-mgr", ""]}) -- 0x7fd9e80008d0 con 0x7fd9d4039050 2024-04-25T01:36:02.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:01.856+0000 7fd9e57fa700 1 -- 172.21.15.78:0/1133071394 <== mgr.14116 v2:172.21.15.78:6800/1101495738 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fd9e80008d0 con 0x7fd9d4039050 2024-04-25T01:36:02.431 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:01.859+0000 7fd9eea4b700 1 -- 172.21.15.78:0/1133071394 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fd9d4039050 msgr2=0x7fd9d403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:02.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:01.859+0000 7fd9eea4b700 1 --2- 172.21.15.78:0/1133071394 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fd9d4039050 0x7fd9d403b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0006fd0 tx=0x7fd9d0006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:02.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:01.859+0000 7fd9eea4b700 1 -- 172.21.15.78:0/1133071394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8106c00 msgr2=0x7fd9e819bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:02.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:01.859+0000 7fd9eea4b700 1 --2- 172.21.15.78:0/1133071394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8106c00 0x7fd9e819bcc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8000c00 tx=0x7fd9d8011c10 comp rx=0 tx=0).stop 2024-04-25T01:36:02.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:01.859+0000 7fd9eea4b700 1 -- 172.21.15.78:0/1133071394 shutdown_connections 2024-04-25T01:36:02.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:01.859+0000 7fd9eea4b700 1 --2- 172.21.15.78:0/1133071394 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fd9d4039050 0x7fd9d403b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:02.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:01.859+0000 7fd9eea4b700 1 --2- 172.21.15.78:0/1133071394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8106c00 0x7fd9e819bcc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:02.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:01.859+0000 7fd9eea4b700 1 -- 172.21.15.78:0/1133071394 >> 172.21.15.78:0/1133071394 conn(0x7fd9e81021a0 msgr2=0x7fd9e8103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:02.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:01.859+0000 7fd9eea4b700 1 -- 172.21.15.78:0/1133071394 shutdown_connections 2024-04-25T01:36:02.432 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:01.859+0000 7fd9eea4b700 1 -- 172.21.15.78:0/1133071394 wait complete. 2024-04-25T01:36:02.432 INFO:teuthology.orchestra.run.smithi078.stdout:Deploying unmanaged mon service... 2024-04-25T01:36:02.948 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:02 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' 2024-04-25T01:36:02.948 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:02 smithi078 ceph-mon[29469]: Added host smithi078 2024-04-25T01:36:02.949 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:02 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:36:03.920 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T01:36:03.920 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.016+0000 7f0be9a33700 1 Processor -- start 2024-04-25T01:36:03.920 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.016+0000 7f0be9a33700 1 -- start start 2024-04-25T01:36:03.920 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.016+0000 7f0be9a33700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be4106bf0 0x7f0be4107010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:03.920 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.016+0000 7f0be9a33700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0be41075e0 con 0x7f0be4106bf0 2024-04-25T01:36:03.920 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.017+0000 7f0be2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be4106bf0 0x7f0be4107010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:03.920 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.017+0000 7f0be2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be4106bf0 0x7f0be4107010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39166/0 (socket says 172.21.15.78:39166) 2024-04-25T01:36:03.920 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.017+0000 7f0be2ffd700 1 -- 172.21.15.78:0/2507025681 learned_addr learned my addr 172.21.15.78:0/2507025681 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:03.920 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.017+0000 7f0be2ffd700 1 -- 172.21.15.78:0/2507025681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be4107df0 con 0x7f0be4106bf0 2024-04-25T01:36:03.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.017+0000 7f0be2ffd700 1 --2- 172.21.15.78:0/2507025681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be4106bf0 0x7f0be4107010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40098a0 tx=0x7f0bd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fbedd00787e917 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:03.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.017+0000 7f0be1ffb700 1 -- 172.21.15.78:0/2507025681 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bd4004160 con 0x7f0be4106bf0 2024-04-25T01:36:03.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.017+0000 7f0be1ffb700 1 -- 172.21.15.78:0/2507025681 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0bd4004300 con 0x7f0be4106bf0 2024-04-25T01:36:03.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.017+0000 7f0be1ffb700 1 -- 172.21.15.78:0/2507025681 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bd4011b50 con 0x7f0be4106bf0 2024-04-25T01:36:03.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.018+0000 7f0be9a33700 1 -- 172.21.15.78:0/2507025681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be4106bf0 msgr2=0x7f0be4107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:03.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.018+0000 7f0be9a33700 1 --2- 172.21.15.78:0/2507025681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be4106bf0 0x7f0be4107010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40098a0 tx=0x7f0bd4009bb0 comp rx=0 tx=0).stop 2024-04-25T01:36:03.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.018+0000 7f0be9a33700 1 -- 172.21.15.78:0/2507025681 shutdown_connections 2024-04-25T01:36:03.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.018+0000 7f0be9a33700 1 --2- 172.21.15.78:0/2507025681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be4106bf0 0x7f0be4107010 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:03.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.018+0000 7f0be9a33700 1 -- 172.21.15.78:0/2507025681 >> 172.21.15.78:0/2507025681 conn(0x7f0be41021b0 msgr2=0x7f0be41045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:03.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.018+0000 7f0be9a33700 1 -- 172.21.15.78:0/2507025681 shutdown_connections 2024-04-25T01:36:03.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.018+0000 7f0be9a33700 1 -- 172.21.15.78:0/2507025681 wait complete. 2024-04-25T01:36:03.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.018+0000 7f0be9a33700 1 Processor -- start 2024-04-25T01:36:03.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.019+0000 7f0be9a33700 1 -- start start 2024-04-25T01:36:03.921 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.019+0000 7f0be9a33700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be4106bf0 0x7f0be419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:03.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.019+0000 7f0be9a33700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0be410b1a0 con 0x7f0be4106bf0 2024-04-25T01:36:03.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.019+0000 7f0be2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be4106bf0 0x7f0be419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:03.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.019+0000 7f0be2ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be4106bf0 0x7f0be419bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39178/0 (socket says 172.21.15.78:39178) 2024-04-25T01:36:03.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.019+0000 7f0be2ffd700 1 -- 172.21.15.78:0/2001491158 learned_addr learned my addr 172.21.15.78:0/2001491158 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:03.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.019+0000 7f0be2ffd700 1 -- 172.21.15.78:0/2001491158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be419c2a0 con 0x7f0be4106bf0 2024-04-25T01:36:03.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.019+0000 7f0be2ffd700 1 --2- 172.21.15.78:0/2001491158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be4106bf0 0x7f0be419bd60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4000c00 tx=0x7f0bd4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:03.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.019+0000 7f0be8a31700 1 -- 172.21.15.78:0/2001491158 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bd4003b20 con 0x7f0be4106bf0 2024-04-25T01:36:03.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.019+0000 7f0be8a31700 1 -- 172.21.15.78:0/2001491158 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0bd4003cc0 con 0x7f0be4106bf0 2024-04-25T01:36:03.922 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.019+0000 7f0be8a31700 1 -- 172.21.15.78:0/2001491158 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bd4024910 con 0x7f0be4106bf0 2024-04-25T01:36:03.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.019+0000 7f0be9a33700 1 -- 172.21.15.78:0/2001491158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be419c530 con 0x7f0be4106bf0 2024-04-25T01:36:03.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.020+0000 7f0be9a33700 1 -- 172.21.15.78:0/2001491158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be419f1c0 con 0x7f0be4106bf0 2024-04-25T01:36:03.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.020+0000 7f0be8a31700 1 -- 172.21.15.78:0/2001491158 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0bd4022020 con 0x7f0be4106bf0 2024-04-25T01:36:03.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.020+0000 7f0be9a33700 1 -- 172.21.15.78:0/2001491158 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be4063890 con 0x7f0be4106bf0 2024-04-25T01:36:03.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.021+0000 7f0be8a31700 1 --2- 172.21.15.78:0/2001491158 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f0bd0039050 0x7f0bd003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:03.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.021+0000 7f0be8a31700 1 -- 172.21.15.78:0/2001491158 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0bd4011e00 con 0x7f0be4106bf0 2024-04-25T01:36:03.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.021+0000 7f0be27fc700 1 --2- 172.21.15.78:0/2001491158 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f0bd0039050 0x7f0bd003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:03.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.021+0000 7f0be27fc700 1 --2- 172.21.15.78:0/2001491158 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f0bd0039050 0x7f0bd003b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc006fd0 tx=0x7f0bcc006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:03.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.026+0000 7f0be8a31700 1 -- 172.21.15.78:0/2001491158 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0bd402cdf0 con 0x7f0be4106bf0 2024-04-25T01:36:03.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.264+0000 7f0be9a33700 1 -- 172.21.15.78:0/2001491158 --> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0be40008d0 con 0x7f0bd0039050 2024-04-25T01:36:03.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.269+0000 7f0be8a31700 1 -- 172.21.15.78:0/2001491158 <== mgr.14116 v2:172.21.15.78:6800/1101495738 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0be40008d0 con 0x7f0bd0039050 2024-04-25T01:36:03.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.271+0000 7f0be9a33700 1 -- 172.21.15.78:0/2001491158 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f0bd0039050 msgr2=0x7f0bd003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:03.923 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.271+0000 7f0be9a33700 1 --2- 172.21.15.78:0/2001491158 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f0bd0039050 0x7f0bd003b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc006fd0 tx=0x7f0bcc006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:03.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.271+0000 7f0be9a33700 1 -- 172.21.15.78:0/2001491158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be4106bf0 msgr2=0x7f0be419bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:03.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.271+0000 7f0be9a33700 1 --2- 172.21.15.78:0/2001491158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be4106bf0 0x7f0be419bd60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4000c00 tx=0x7f0bd4011c10 comp rx=0 tx=0).stop 2024-04-25T01:36:03.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.271+0000 7f0be9a33700 1 -- 172.21.15.78:0/2001491158 shutdown_connections 2024-04-25T01:36:03.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.271+0000 7f0be9a33700 1 --2- 172.21.15.78:0/2001491158 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f0bd0039050 0x7f0bd003b510 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:03.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.271+0000 7f0be9a33700 1 --2- 172.21.15.78:0/2001491158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0be4106bf0 0x7f0be419bd60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:03.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.271+0000 7f0be9a33700 1 -- 172.21.15.78:0/2001491158 >> 172.21.15.78:0/2001491158 conn(0x7f0be41021b0 msgr2=0x7f0be4103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:03.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.271+0000 7f0be9a33700 1 -- 172.21.15.78:0/2001491158 shutdown_connections 2024-04-25T01:36:03.924 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:03.271+0000 7f0be9a33700 1 -- 172.21.15.78:0/2001491158 wait complete. 2024-04-25T01:36:03.924 INFO:teuthology.orchestra.run.smithi078.stdout:Deploying unmanaged mgr service... 2024-04-25T01:36:04.491 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:04 smithi078 ceph-mon[29469]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:04.491 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:04 smithi078 ceph-mon[29469]: Saving service mon spec with placement count:5 2024-04-25T01:36:04.491 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:04 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' 2024-04-25T01:36:05.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T01:36:05.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.369+0000 7f05c7677700 1 Processor -- start 2024-04-25T01:36:05.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.369+0000 7f05c7677700 1 -- start start 2024-04-25T01:36:05.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.370+0000 7f05c7677700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c01087a0 0x7f05c010ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:05.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.370+0000 7f05c7677700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05c007be30 con 0x7f05c01087a0 2024-04-25T01:36:05.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.370+0000 7f05c5413700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c01087a0 0x7f05c010ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:05.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.370+0000 7f05c5413700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c01087a0 0x7f05c010ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39182/0 (socket says 172.21.15.78:39182) 2024-04-25T01:36:05.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.370+0000 7f05c5413700 1 -- 172.21.15.78:0/2231642551 learned_addr learned my addr 172.21.15.78:0/2231642551 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:05.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.370+0000 7f05c5413700 1 -- 172.21.15.78:0/2231642551 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c010b0d0 con 0x7f05c01087a0 2024-04-25T01:36:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.370+0000 7f05c5413700 1 --2- 172.21.15.78:0/2231642551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c01087a0 0x7f05c010ab90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009e80 tx=0x7f05b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97b90d6d8d8ae99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.370+0000 7f05bffff700 1 -- 172.21.15.78:0/2231642551 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05b0017070 con 0x7f05c01087a0 2024-04-25T01:36:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.370+0000 7f05bffff700 1 -- 172.21.15.78:0/2231642551 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f05b000b8c0 con 0x7f05c01087a0 2024-04-25T01:36:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.370+0000 7f05bffff700 1 -- 172.21.15.78:0/2231642551 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05b00045c0 con 0x7f05c01087a0 2024-04-25T01:36:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.371+0000 7f05c7677700 1 -- 172.21.15.78:0/2231642551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c01087a0 msgr2=0x7f05c010ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.371+0000 7f05c7677700 1 --2- 172.21.15.78:0/2231642551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c01087a0 0x7f05c010ab90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009e80 tx=0x7f05b000b330 comp rx=0 tx=0).stop 2024-04-25T01:36:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.371+0000 7f05c7677700 1 -- 172.21.15.78:0/2231642551 shutdown_connections 2024-04-25T01:36:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.371+0000 7f05c7677700 1 --2- 172.21.15.78:0/2231642551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c01087a0 0x7f05c010ab90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.371+0000 7f05c7677700 1 -- 172.21.15.78:0/2231642551 >> 172.21.15.78:0/2231642551 conn(0x7f05c01021b0 msgr2=0x7f05c01045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.371+0000 7f05c7677700 1 -- 172.21.15.78:0/2231642551 shutdown_connections 2024-04-25T01:36:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.371+0000 7f05c7677700 1 -- 172.21.15.78:0/2231642551 wait complete. 2024-04-25T01:36:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.372+0000 7f05c7677700 1 Processor -- start 2024-04-25T01:36:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.372+0000 7f05c7677700 1 -- start start 2024-04-25T01:36:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.372+0000 7f05c7677700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c01087a0 0x7f05c0108020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.372+0000 7f05c7677700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05c010c0f0 con 0x7f05c01087a0 2024-04-25T01:36:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.372+0000 7f05c5413700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c01087a0 0x7f05c0108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.372+0000 7f05c5413700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c01087a0 0x7f05c0108020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39196/0 (socket says 172.21.15.78:39196) 2024-04-25T01:36:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.372+0000 7f05c5413700 1 -- 172.21.15.78:0/2708600147 learned_addr learned my addr 172.21.15.78:0/2708600147 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.372+0000 7f05c5413700 1 -- 172.21.15.78:0/2708600147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c0108560 con 0x7f05c01087a0 2024-04-25T01:36:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.372+0000 7f05c5413700 1 --2- 172.21.15.78:0/2708600147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c01087a0 0x7f05c0108020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f05b00072a0 tx=0x7f05b000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.372+0000 7f05be7fc700 1 -- 172.21.15.78:0/2708600147 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05b0020020 con 0x7f05c01087a0 2024-04-25T01:36:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.373+0000 7f05c7677700 1 -- 172.21.15.78:0/2708600147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c0106750 con 0x7f05c01087a0 2024-04-25T01:36:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.373+0000 7f05be7fc700 1 -- 172.21.15.78:0/2708600147 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f05b0003b50 con 0x7f05c01087a0 2024-04-25T01:36:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.373+0000 7f05be7fc700 1 -- 172.21.15.78:0/2708600147 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05b0017040 con 0x7f05c01087a0 2024-04-25T01:36:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.373+0000 7f05c7677700 1 -- 172.21.15.78:0/2708600147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c0106c90 con 0x7f05c01087a0 2024-04-25T01:36:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.374+0000 7f05be7fc700 1 -- 172.21.15.78:0/2708600147 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f05b0023020 con 0x7f05c01087a0 2024-04-25T01:36:05.189 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.374+0000 7f05c7677700 1 -- 172.21.15.78:0/2708600147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c0063890 con 0x7f05c01087a0 2024-04-25T01:36:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.374+0000 7f05be7fc700 1 --2- 172.21.15.78:0/2708600147 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f05a8039000 0x7f05a803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.374+0000 7f05be7fc700 1 -- 172.21.15.78:0/2708600147 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f05b000f050 con 0x7f05c01087a0 2024-04-25T01:36:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.374+0000 7f05c4c12700 1 --2- 172.21.15.78:0/2708600147 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f05a8039000 0x7f05a803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.374+0000 7f05c4c12700 1 --2- 172.21.15.78:0/2708600147 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f05a8039000 0x7f05a803b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f05b4006fd0 tx=0x7f05b4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.379+0000 7f05be7fc700 1 -- 172.21.15.78:0/2708600147 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f05b002a020 con 0x7f05c01087a0 2024-04-25T01:36:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.596+0000 7f05c7677700 1 -- 172.21.15.78:0/2708600147 --> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f05c0103420 con 0x7f05a8039000 2024-04-25T01:36:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.601+0000 7f05be7fc700 1 -- 172.21.15.78:0/2708600147 <== mgr.14116 v2:172.21.15.78:6800/1101495738 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f05c0103420 con 0x7f05a8039000 2024-04-25T01:36:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.603+0000 7f05c7677700 1 -- 172.21.15.78:0/2708600147 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f05a8039000 msgr2=0x7f05a803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.603+0000 7f05c7677700 1 --2- 172.21.15.78:0/2708600147 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f05a8039000 0x7f05a803b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f05b4006fd0 tx=0x7f05b4006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.603+0000 7f05c7677700 1 -- 172.21.15.78:0/2708600147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c01087a0 msgr2=0x7f05c0108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.603+0000 7f05c7677700 1 --2- 172.21.15.78:0/2708600147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c01087a0 0x7f05c0108020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f05b00072a0 tx=0x7f05b000bd50 comp rx=0 tx=0).stop 2024-04-25T01:36:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.603+0000 7f05c7677700 1 -- 172.21.15.78:0/2708600147 shutdown_connections 2024-04-25T01:36:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.603+0000 7f05c7677700 1 --2- 172.21.15.78:0/2708600147 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f05a8039000 0x7f05a803b4c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.603+0000 7f05c7677700 1 --2- 172.21.15.78:0/2708600147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05c01087a0 0x7f05c0108020 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.603+0000 7f05c7677700 1 -- 172.21.15.78:0/2708600147 >> 172.21.15.78:0/2708600147 conn(0x7f05c01021b0 msgr2=0x7f05c0102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.603+0000 7f05c7677700 1 -- 172.21.15.78:0/2708600147 shutdown_connections 2024-04-25T01:36:05.191 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:04.603+0000 7f05c7677700 1 -- 172.21.15.78:0/2708600147 wait complete. 2024-04-25T01:36:05.916 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:05 smithi078 ceph-mon[29469]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:05.916 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:05 smithi078 ceph-mon[29469]: Saving service mgr spec with placement count:2 2024-04-25T01:36:05.916 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:05 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' 2024-04-25T01:36:07.130 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.360+0000 7f418c37f700 1 Processor -- start 2024-04-25T01:36:07.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.361+0000 7f418c37f700 1 -- start start 2024-04-25T01:36:07.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.361+0000 7f418c37f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4184108750 0x7f418410ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:07.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.361+0000 7f418c37f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f418407be70 con 0x7f4184108750 2024-04-25T01:36:07.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.361+0000 7f418a11b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4184108750 0x7f418410ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:07.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.361+0000 7f418a11b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4184108750 0x7f418410ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39198/0 (socket says 172.21.15.78:39198) 2024-04-25T01:36:07.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.361+0000 7f418a11b700 1 -- 172.21.15.78:0/196196979 learned_addr learned my addr 172.21.15.78:0/196196979 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:07.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.362+0000 7f418a11b700 1 -- 172.21.15.78:0/196196979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418410b080 con 0x7f4184108750 2024-04-25T01:36:07.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.362+0000 7f418a11b700 1 --2- 172.21.15.78:0/196196979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4184108750 0x7f418410ab40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f41800098a0 tx=0x7f4180009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec28579ee8837556 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:07.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.362+0000 7f4189119700 1 -- 172.21.15.78:0/196196979 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4180004160 con 0x7f4184108750 2024-04-25T01:36:07.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.362+0000 7f4189119700 1 -- 172.21.15.78:0/196196979 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4180004300 con 0x7f4184108750 2024-04-25T01:36:07.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.362+0000 7f4189119700 1 -- 172.21.15.78:0/196196979 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4180011b20 con 0x7f4184108750 2024-04-25T01:36:07.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.363+0000 7f418c37f700 1 -- 172.21.15.78:0/196196979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4184108750 msgr2=0x7f418410ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:07.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.363+0000 7f418c37f700 1 --2- 172.21.15.78:0/196196979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4184108750 0x7f418410ab40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f41800098a0 tx=0x7f4180009bb0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.363+0000 7f418c37f700 1 -- 172.21.15.78:0/196196979 shutdown_connections 2024-04-25T01:36:07.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.363+0000 7f418c37f700 1 --2- 172.21.15.78:0/196196979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4184108750 0x7f418410ab40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.363+0000 7f418c37f700 1 -- 172.21.15.78:0/196196979 >> 172.21.15.78:0/196196979 conn(0x7f4184102180 msgr2=0x7f41841045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:07.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.363+0000 7f418c37f700 1 -- 172.21.15.78:0/196196979 shutdown_connections 2024-04-25T01:36:07.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.363+0000 7f418c37f700 1 -- 172.21.15.78:0/196196979 wait complete. 2024-04-25T01:36:07.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.364+0000 7f418c37f700 1 Processor -- start 2024-04-25T01:36:07.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.364+0000 7f418c37f700 1 -- start start 2024-04-25T01:36:07.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.364+0000 7f418c37f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4184108750 0x7f4184107fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:07.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.364+0000 7f418c37f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f418410c0f0 con 0x7f4184108750 2024-04-25T01:36:07.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.364+0000 7f418a11b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4184108750 0x7f4184107fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:07.132 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.364+0000 7f418a11b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4184108750 0x7f4184107fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39208/0 (socket says 172.21.15.78:39208) 2024-04-25T01:36:07.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.364+0000 7f418a11b700 1 -- 172.21.15.78:0/1665119836 learned_addr learned my addr 172.21.15.78:0/1665119836 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:07.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.364+0000 7f418a11b700 1 -- 172.21.15.78:0/1665119836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4184108510 con 0x7f4184108750 2024-04-25T01:36:07.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.364+0000 7f418a11b700 1 --2- 172.21.15.78:0/1665119836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4184108750 0x7f4184107fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f41800117e0 tx=0x7f4180011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:07.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.365+0000 7f417b7fe700 1 -- 172.21.15.78:0/1665119836 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4180003ba0 con 0x7f4184108750 2024-04-25T01:36:07.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.365+0000 7f417b7fe700 1 -- 172.21.15.78:0/1665119836 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f418001a910 con 0x7f4184108750 2024-04-25T01:36:07.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.365+0000 7f418c37f700 1 -- 172.21.15.78:0/1665119836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4184106700 con 0x7f4184108750 2024-04-25T01:36:07.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.365+0000 7f417b7fe700 1 -- 172.21.15.78:0/1665119836 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41800248f0 con 0x7f4184108750 2024-04-25T01:36:07.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.365+0000 7f418c37f700 1 -- 172.21.15.78:0/1665119836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4184106bc0 con 0x7f4184108750 2024-04-25T01:36:07.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.366+0000 7f418c37f700 1 -- 172.21.15.78:0/1665119836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4184063890 con 0x7f4184108750 2024-04-25T01:36:07.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.366+0000 7f417b7fe700 1 -- 172.21.15.78:0/1665119836 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4180022020 con 0x7f4184108750 2024-04-25T01:36:07.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.366+0000 7f417b7fe700 1 --2- 172.21.15.78:0/1665119836 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f41700390a0 0x7f417003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:07.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.366+0000 7f417b7fe700 1 -- 172.21.15.78:0/1665119836 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f418004d440 con 0x7f4184108750 2024-04-25T01:36:07.133 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.367+0000 7f418991a700 1 --2- 172.21.15.78:0/1665119836 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f41700390a0 0x7f417003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:07.134 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.367+0000 7f418991a700 1 --2- 172.21.15.78:0/1665119836 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f41700390a0 0x7f417003b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4174006fd0 tx=0x7f4174006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:07.134 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.371+0000 7f417b7fe700 1 -- 172.21.15.78:0/1665119836 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4180016260 con 0x7f4184108750 2024-04-25T01:36:07.134 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.587+0000 7f418c37f700 1 -- 172.21.15.78:0/1665119836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f41841072e0 con 0x7f4184108750 2024-04-25T01:36:07.134 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.591+0000 7f417b7fe700 1 -- 172.21.15.78:0/1665119836 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f418002ddb0 con 0x7f4184108750 2024-04-25T01:36:07.134 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.594+0000 7f418c37f700 1 -- 172.21.15.78:0/1665119836 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f41700390a0 msgr2=0x7f417003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:07.134 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.594+0000 7f418c37f700 1 --2- 172.21.15.78:0/1665119836 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f41700390a0 0x7f417003b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4174006fd0 tx=0x7f4174006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:07.134 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.594+0000 7f418c37f700 1 -- 172.21.15.78:0/1665119836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4184108750 msgr2=0x7f4184107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:07.134 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.594+0000 7f418c37f700 1 --2- 172.21.15.78:0/1665119836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4184108750 0x7f4184107fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f41800117e0 tx=0x7f4180011ba0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.134 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.594+0000 7f418c37f700 1 -- 172.21.15.78:0/1665119836 shutdown_connections 2024-04-25T01:36:07.134 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.594+0000 7f418c37f700 1 --2- 172.21.15.78:0/1665119836 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f41700390a0 0x7f417003b560 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.134 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.594+0000 7f418c37f700 1 --2- 172.21.15.78:0/1665119836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4184108750 0x7f4184107fd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.134 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.594+0000 7f418c37f700 1 -- 172.21.15.78:0/1665119836 >> 172.21.15.78:0/1665119836 conn(0x7f4184102180 msgr2=0x7f4184102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:07.134 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.594+0000 7f418c37f700 1 -- 172.21.15.78:0/1665119836 shutdown_connections 2024-04-25T01:36:07.135 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:06.594+0000 7f418c37f700 1 -- 172.21.15.78:0/1665119836 wait complete. 2024-04-25T01:36:07.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:07 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' 2024-04-25T01:36:07.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:07 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' 2024-04-25T01:36:07.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:07 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1665119836' entity='client.admin' 2024-04-25T01:36:08.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.603+0000 7f317ad19700 1 Processor -- start 2024-04-25T01:36:08.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.603+0000 7f317ad19700 1 -- start start 2024-04-25T01:36:08.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.603+0000 7f317ad19700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174107d60 0x7f3174108180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:08.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.603+0000 7f317ad19700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3174108750 con 0x7f3174107d60 2024-04-25T01:36:08.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.604+0000 7f3178ab5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174107d60 0x7f3174108180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:08.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.604+0000 7f3178ab5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174107d60 0x7f3174108180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39212/0 (socket says 172.21.15.78:39212) 2024-04-25T01:36:08.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.604+0000 7f3178ab5700 1 -- 172.21.15.78:0/622411115 learned_addr learned my addr 172.21.15.78:0/622411115 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:08.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.604+0000 7f3178ab5700 1 -- 172.21.15.78:0/622411115 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3174108ec0 con 0x7f3174107d60 2024-04-25T01:36:08.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.604+0000 7f3178ab5700 1 --2- 172.21.15.78:0/622411115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174107d60 0x7f3174108180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f31700098a0 tx=0x7f3170009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5c9188dd3d8612 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:08.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.604+0000 7f316b7fe700 1 -- 172.21.15.78:0/622411115 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3170004160 con 0x7f3174107d60 2024-04-25T01:36:08.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.604+0000 7f316b7fe700 1 -- 172.21.15.78:0/622411115 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3170004300 con 0x7f3174107d60 2024-04-25T01:36:08.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.604+0000 7f316b7fe700 1 -- 172.21.15.78:0/622411115 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3170011b20 con 0x7f3174107d60 2024-04-25T01:36:08.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.605+0000 7f317ad19700 1 -- 172.21.15.78:0/622411115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174107d60 msgr2=0x7f3174108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:08.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.605+0000 7f317ad19700 1 --2- 172.21.15.78:0/622411115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174107d60 0x7f3174108180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f31700098a0 tx=0x7f3170009bb0 comp rx=0 tx=0).stop 2024-04-25T01:36:08.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.605+0000 7f317ad19700 1 -- 172.21.15.78:0/622411115 shutdown_connections 2024-04-25T01:36:08.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.605+0000 7f317ad19700 1 --2- 172.21.15.78:0/622411115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174107d60 0x7f3174108180 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:08.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.605+0000 7f317ad19700 1 -- 172.21.15.78:0/622411115 >> 172.21.15.78:0/622411115 conn(0x7f3174103300 msgr2=0x7f3174105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:08.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.605+0000 7f317ad19700 1 -- 172.21.15.78:0/622411115 shutdown_connections 2024-04-25T01:36:08.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.605+0000 7f317ad19700 1 -- 172.21.15.78:0/622411115 wait complete. 2024-04-25T01:36:08.822 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.605+0000 7f317ad19700 1 Processor -- start 2024-04-25T01:36:08.822 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.605+0000 7f317ad19700 1 -- start start 2024-04-25T01:36:08.822 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.606+0000 7f317ad19700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174107d60 0x7f317419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:08.822 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.606+0000 7f317ad19700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f317410c270 con 0x7f3174107d60 2024-04-25T01:36:08.822 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.606+0000 7f3178ab5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174107d60 0x7f317419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:08.822 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.606+0000 7f3178ab5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174107d60 0x7f317419bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39222/0 (socket says 172.21.15.78:39222) 2024-04-25T01:36:08.823 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.606+0000 7f3178ab5700 1 -- 172.21.15.78:0/3336573148 learned_addr learned my addr 172.21.15.78:0/3336573148 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:08.823 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.606+0000 7f3178ab5700 1 -- 172.21.15.78:0/3336573148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317419c170 con 0x7f3174107d60 2024-04-25T01:36:08.823 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.606+0000 7f3178ab5700 1 --2- 172.21.15.78:0/3336573148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174107d60 0x7f317419bc30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f31700117e0 tx=0x7f3170011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:08.823 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.606+0000 7f3169ffb700 1 -- 172.21.15.78:0/3336573148 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3170003bf0 con 0x7f3174107d60 2024-04-25T01:36:08.823 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.606+0000 7f3169ffb700 1 -- 172.21.15.78:0/3336573148 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f317001a910 con 0x7f3174107d60 2024-04-25T01:36:08.823 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.606+0000 7f317ad19700 1 -- 172.21.15.78:0/3336573148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317419c400 con 0x7f3174107d60 2024-04-25T01:36:08.823 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.606+0000 7f3169ffb700 1 -- 172.21.15.78:0/3336573148 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3170024970 con 0x7f3174107d60 2024-04-25T01:36:08.823 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.606+0000 7f317ad19700 1 -- 172.21.15.78:0/3336573148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317419c860 con 0x7f3174107d60 2024-04-25T01:36:08.824 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.607+0000 7f3169ffb700 1 -- 172.21.15.78:0/3336573148 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3170022020 con 0x7f3174107d60 2024-04-25T01:36:08.824 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.607+0000 7f317ad19700 1 -- 172.21.15.78:0/3336573148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3174063890 con 0x7f3174107d60 2024-04-25T01:36:08.824 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.607+0000 7f3169ffb700 1 --2- 172.21.15.78:0/3336573148 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f315c039050 0x7f315c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:08.824 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.607+0000 7f3169ffb700 1 -- 172.21.15.78:0/3336573148 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f317004d840 con 0x7f3174107d60 2024-04-25T01:36:08.824 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.608+0000 7f316bfff700 1 --2- 172.21.15.78:0/3336573148 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f315c039050 0x7f315c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:08.824 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.608+0000 7f316bfff700 1 --2- 172.21.15.78:0/3336573148 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f315c039050 0x7f315c03b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3164006fd0 tx=0x7f3164006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:08.825 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.613+0000 7f3169ffb700 1 -- 172.21.15.78:0/3336573148 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f317002cdf0 con 0x7f3174107d60 2024-04-25T01:36:08.825 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.829+0000 7f317ad19700 1 -- 172.21.15.78:0/3336573148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f317419f2d0 con 0x7f3174107d60 2024-04-25T01:36:08.825 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.832+0000 7f3169ffb700 1 -- 172.21.15.78:0/3336573148 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f3170020c10 con 0x7f3174107d60 2024-04-25T01:36:08.825 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.834+0000 7f317ad19700 1 -- 172.21.15.78:0/3336573148 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f315c039050 msgr2=0x7f315c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:08.825 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.834+0000 7f317ad19700 1 --2- 172.21.15.78:0/3336573148 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f315c039050 0x7f315c03b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3164006fd0 tx=0x7f3164006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:08.825 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.834+0000 7f317ad19700 1 -- 172.21.15.78:0/3336573148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174107d60 msgr2=0x7f317419bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:08.825 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.834+0000 7f317ad19700 1 --2- 172.21.15.78:0/3336573148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174107d60 0x7f317419bc30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f31700117e0 tx=0x7f3170011ba0 comp rx=0 tx=0).stop 2024-04-25T01:36:08.825 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.834+0000 7f317ad19700 1 -- 172.21.15.78:0/3336573148 shutdown_connections 2024-04-25T01:36:08.826 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.834+0000 7f317ad19700 1 --2- 172.21.15.78:0/3336573148 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f315c039050 0x7f315c03b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:08.826 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.834+0000 7f317ad19700 1 --2- 172.21.15.78:0/3336573148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3174107d60 0x7f317419bc30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:08.826 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.834+0000 7f317ad19700 1 -- 172.21.15.78:0/3336573148 >> 172.21.15.78:0/3336573148 conn(0x7f3174103300 msgr2=0x7f3174103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:08.826 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.835+0000 7f317ad19700 1 -- 172.21.15.78:0/3336573148 shutdown_connections 2024-04-25T01:36:08.826 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:07.835+0000 7f317ad19700 1 -- 172.21.15.78:0/3336573148 wait complete. 2024-04-25T01:36:08.826 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling the dashboard module... 2024-04-25T01:36:08.933 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:08 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3336573148' entity='client.admin' 2024-04-25T01:36:10.156 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:09 smithi078 ceph-mon[29469]: from='mgr.14116 172.21.15.78:0/3548838998' entity='mgr.x' 2024-04-25T01:36:10.156 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:09 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/4136949295' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T01:36:10.157 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:09 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: ignoring --setuser ceph since I am not root 2024-04-25T01:36:10.157 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:09 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: ignoring --setgroup ceph since I am not root 2024-04-25T01:36:10.157 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:10 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:10.097+0000 7f429184b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:36:10.472 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:10 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:10.288+0000 7f429184b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:36:10.472 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:10 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:10.389+0000 7f429184b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:36:10.480 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.270+0000 7ff269488700 1 Processor -- start 2024-04-25T01:36:10.480 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.270+0000 7ff269488700 1 -- start start 2024-04-25T01:36:10.480 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.270+0000 7ff269488700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff26407e690 0x7ff26407cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:10.480 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.270+0000 7ff269488700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff26407d0d0 con 0x7ff26407e690 2024-04-25T01:36:10.480 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.270+0000 7ff262ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff26407e690 0x7ff26407cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:10.480 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.270+0000 7ff262ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff26407e690 0x7ff26407cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39228/0 (socket says 172.21.15.78:39228) 2024-04-25T01:36:10.480 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.270+0000 7ff262ffd700 1 -- 172.21.15.78:0/1757491785 learned_addr learned my addr 172.21.15.78:0/1757491785 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:10.480 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.271+0000 7ff262ffd700 1 -- 172.21.15.78:0/1757491785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26407d250 con 0x7ff26407e690 2024-04-25T01:36:10.480 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.271+0000 7ff262ffd700 1 --2- 172.21.15.78:0/1757491785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff26407e690 0x7ff26407cb90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff2540098a0 tx=0x7ff254009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2973670e0aca04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:10.481 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.271+0000 7ff261ffb700 1 -- 172.21.15.78:0/1757491785 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff254004160 con 0x7ff26407e690 2024-04-25T01:36:10.481 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.271+0000 7ff261ffb700 1 -- 172.21.15.78:0/1757491785 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff254004300 con 0x7ff26407e690 2024-04-25T01:36:10.481 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.271+0000 7ff261ffb700 1 -- 172.21.15.78:0/1757491785 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff254011b20 con 0x7ff26407e690 2024-04-25T01:36:10.481 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.272+0000 7ff269488700 1 -- 172.21.15.78:0/1757491785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff26407e690 msgr2=0x7ff26407cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:10.481 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.272+0000 7ff269488700 1 --2- 172.21.15.78:0/1757491785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff26407e690 0x7ff26407cb90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff2540098a0 tx=0x7ff254009bb0 comp rx=0 tx=0).stop 2024-04-25T01:36:10.481 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.272+0000 7ff269488700 1 -- 172.21.15.78:0/1757491785 shutdown_connections 2024-04-25T01:36:10.481 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.272+0000 7ff269488700 1 --2- 172.21.15.78:0/1757491785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff26407e690 0x7ff26407cb90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:10.481 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.272+0000 7ff269488700 1 -- 172.21.15.78:0/1757491785 >> 172.21.15.78:0/1757491785 conn(0x7ff264103300 msgr2=0x7ff264105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:10.481 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.272+0000 7ff269488700 1 -- 172.21.15.78:0/1757491785 shutdown_connections 2024-04-25T01:36:10.481 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.272+0000 7ff269488700 1 -- 172.21.15.78:0/1757491785 wait complete. 2024-04-25T01:36:10.481 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.272+0000 7ff269488700 1 Processor -- start 2024-04-25T01:36:10.481 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.272+0000 7ff269488700 1 -- start start 2024-04-25T01:36:10.482 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.273+0000 7ff269488700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff26407e690 0x7ff26419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:10.482 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.273+0000 7ff269488700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff26410e2c0 con 0x7ff26407e690 2024-04-25T01:36:10.482 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.273+0000 7ff262ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff26407e690 0x7ff26419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:10.482 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.273+0000 7ff262ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff26407e690 0x7ff26419ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39240/0 (socket says 172.21.15.78:39240) 2024-04-25T01:36:10.482 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.273+0000 7ff262ffd700 1 -- 172.21.15.78:0/4136949295 learned_addr learned my addr 172.21.15.78:0/4136949295 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:10.482 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.273+0000 7ff262ffd700 1 -- 172.21.15.78:0/4136949295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26419e2f0 con 0x7ff26407e690 2024-04-25T01:36:10.482 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.273+0000 7ff262ffd700 1 --2- 172.21.15.78:0/4136949295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff26407e690 0x7ff26419ddb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff2540117e0 tx=0x7ff254011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:10.482 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.273+0000 7ff24bfff700 1 -- 172.21.15.78:0/4136949295 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff254003ba0 con 0x7ff26407e690 2024-04-25T01:36:10.482 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.273+0000 7ff24bfff700 1 -- 172.21.15.78:0/4136949295 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff25401a910 con 0x7ff26407e690 2024-04-25T01:36:10.482 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.273+0000 7ff269488700 1 -- 172.21.15.78:0/4136949295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26419e580 con 0x7ff26407e690 2024-04-25T01:36:10.483 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.273+0000 7ff269488700 1 -- 172.21.15.78:0/4136949295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26419e9e0 con 0x7ff26407e690 2024-04-25T01:36:10.483 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.274+0000 7ff24bfff700 1 -- 172.21.15.78:0/4136949295 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2540248f0 con 0x7ff26407e690 2024-04-25T01:36:10.483 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.274+0000 7ff24bfff700 1 -- 172.21.15.78:0/4136949295 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff254022020 con 0x7ff26407e690 2024-04-25T01:36:10.483 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.275+0000 7ff24bfff700 1 --2- 172.21.15.78:0/4136949295 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff24c038c60 0x7ff24c03b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:10.483 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.275+0000 7ff269488700 1 -- 172.21.15.78:0/4136949295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff250005360 con 0x7ff26407e690 2024-04-25T01:36:10.483 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.275+0000 7ff24bfff700 1 -- 172.21.15.78:0/4136949295 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff25404d920 con 0x7ff26407e690 2024-04-25T01:36:10.483 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.275+0000 7ff2627fc700 1 --2- 172.21.15.78:0/4136949295 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff24c038c60 0x7ff24c03b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:10.483 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.275+0000 7ff2627fc700 1 --2- 172.21.15.78:0/4136949295 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff24c038c60 0x7ff24c03b120 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff258006fd0 tx=0x7ff258006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:10.483 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.280+0000 7ff24bfff700 1 -- 172.21.15.78:0/4136949295 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff25400b3e0 con 0x7ff26407e690 2024-04-25T01:36:10.483 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.546+0000 7ff269488700 1 -- 172.21.15.78:0/4136949295 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7ff250005650 con 0x7ff26407e690 2024-04-25T01:36:10.483 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.912+0000 7ff24bfff700 1 -- 172.21.15.78:0/4136949295 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7ff254020c10 con 0x7ff26407e690 2024-04-25T01:36:10.483 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.914+0000 7ff24bfff700 1 -- 172.21.15.78:0/4136949295 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7ff25402dc60 con 0x7ff26407e690 2024-04-25T01:36:10.483 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.915+0000 7ff269488700 1 -- 172.21.15.78:0/4136949295 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff24c038c60 msgr2=0x7ff24c03b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:10.484 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.915+0000 7ff269488700 1 --2- 172.21.15.78:0/4136949295 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff24c038c60 0x7ff24c03b120 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff258006fd0 tx=0x7ff258006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:10.484 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.915+0000 7ff269488700 1 -- 172.21.15.78:0/4136949295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff26407e690 msgr2=0x7ff26419ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:10.484 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.915+0000 7ff269488700 1 --2- 172.21.15.78:0/4136949295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff26407e690 0x7ff26419ddb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff2540117e0 tx=0x7ff254011ba0 comp rx=0 tx=0).stop 2024-04-25T01:36:10.484 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.915+0000 7ff269488700 1 -- 172.21.15.78:0/4136949295 shutdown_connections 2024-04-25T01:36:10.484 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.915+0000 7ff269488700 1 --2- 172.21.15.78:0/4136949295 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7ff24c038c60 0x7ff24c03b120 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:10.484 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.915+0000 7ff269488700 1 --2- 172.21.15.78:0/4136949295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff26407e690 0x7ff26419ddb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:10.484 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.915+0000 7ff269488700 1 -- 172.21.15.78:0/4136949295 >> 172.21.15.78:0/4136949295 conn(0x7ff264103300 msgr2=0x7ff264103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:10.484 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.915+0000 7ff269488700 1 -- 172.21.15.78:0/4136949295 shutdown_connections 2024-04-25T01:36:10.484 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:09.915+0000 7ff269488700 1 -- 172.21.15.78:0/4136949295 wait complete. 2024-04-25T01:36:10.723 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:10 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:10.713+0000 7f429184b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:36:10.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:10 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/4136949295' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T01:36:10.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:10 smithi078 ceph-mon[29469]: mgrmap e9: x(active, since 19s) 2024-04-25T01:36:10.991 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:10 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:10.802+0000 7f429184b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:36:11.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:36:11.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T01:36:11.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T01:36:11.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T01:36:11.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T01:36:11.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:36:11.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.950+0000 7f2537034700 1 Processor -- start 2024-04-25T01:36:11.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.950+0000 7f2537034700 1 -- start start 2024-04-25T01:36:11.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.950+0000 7f2537034700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2530107da0 0x7f25301081c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:11.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.950+0000 7f2537034700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2530108790 con 0x7f2530107da0 2024-04-25T01:36:11.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.951+0000 7f2534dd0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2530107da0 0x7f25301081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:11.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.951+0000 7f2534dd0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2530107da0 0x7f25301081c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56260/0 (socket says 172.21.15.78:56260) 2024-04-25T01:36:11.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.951+0000 7f2534dd0700 1 -- 172.21.15.78:0/4029731320 learned_addr learned my addr 172.21.15.78:0/4029731320 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:11.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.951+0000 7f2534dd0700 1 -- 172.21.15.78:0/4029731320 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2530108f00 con 0x7f2530107da0 2024-04-25T01:36:11.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.951+0000 7f2534dd0700 1 --2- 172.21.15.78:0/4029731320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2530107da0 0x7f25301081c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f25240098a0 tx=0x7f2524009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3181e4cfcb9212 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:11.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.951+0000 7f252f7fe700 1 -- 172.21.15.78:0/4029731320 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2524004160 con 0x7f2530107da0 2024-04-25T01:36:11.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.951+0000 7f252f7fe700 1 -- 172.21.15.78:0/4029731320 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2524004300 con 0x7f2530107da0 2024-04-25T01:36:11.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.951+0000 7f252f7fe700 1 -- 172.21.15.78:0/4029731320 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2524011b20 con 0x7f2530107da0 2024-04-25T01:36:11.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.952+0000 7f2537034700 1 -- 172.21.15.78:0/4029731320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2530107da0 msgr2=0x7f25301081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:11.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.952+0000 7f2537034700 1 --2- 172.21.15.78:0/4029731320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2530107da0 0x7f25301081c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f25240098a0 tx=0x7f2524009bb0 comp rx=0 tx=0).stop 2024-04-25T01:36:11.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.952+0000 7f2537034700 1 -- 172.21.15.78:0/4029731320 shutdown_connections 2024-04-25T01:36:11.747 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.952+0000 7f2537034700 1 --2- 172.21.15.78:0/4029731320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2530107da0 0x7f25301081c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:11.747 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.952+0000 7f2537034700 1 -- 172.21.15.78:0/4029731320 >> 172.21.15.78:0/4029731320 conn(0x7f2530103360 msgr2=0x7f2530105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:11.747 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.952+0000 7f2537034700 1 -- 172.21.15.78:0/4029731320 shutdown_connections 2024-04-25T01:36:11.747 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.952+0000 7f2537034700 1 -- 172.21.15.78:0/4029731320 wait complete. 2024-04-25T01:36:11.747 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.952+0000 7f2537034700 1 Processor -- start 2024-04-25T01:36:11.747 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.953+0000 7f2537034700 1 -- start start 2024-04-25T01:36:11.747 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.953+0000 7f2537034700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2530107da0 0x7f253019bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:11.747 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.953+0000 7f2537034700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f253010c2b0 con 0x7f2530107da0 2024-04-25T01:36:11.747 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.953+0000 7f2534dd0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2530107da0 0x7f253019bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:11.747 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.953+0000 7f2534dd0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2530107da0 0x7f253019bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56276/0 (socket says 172.21.15.78:56276) 2024-04-25T01:36:11.747 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.953+0000 7f2534dd0700 1 -- 172.21.15.78:0/2879244460 learned_addr learned my addr 172.21.15.78:0/2879244460 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:11.747 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.953+0000 7f2534dd0700 1 -- 172.21.15.78:0/2879244460 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253019c180 con 0x7f2530107da0 2024-04-25T01:36:11.747 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.953+0000 7f2534dd0700 1 --2- 172.21.15.78:0/2879244460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2530107da0 0x7f253019bc40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25240117e0 tx=0x7f2524011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.953+0000 7f252dffb700 1 -- 172.21.15.78:0/2879244460 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2524003bf0 con 0x7f2530107da0 2024-04-25T01:36:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.953+0000 7f2537034700 1 -- 172.21.15.78:0/2879244460 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253019c410 con 0x7f2530107da0 2024-04-25T01:36:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.953+0000 7f252dffb700 1 -- 172.21.15.78:0/2879244460 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f252401a910 con 0x7f2530107da0 2024-04-25T01:36:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.953+0000 7f252dffb700 1 -- 172.21.15.78:0/2879244460 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2524024970 con 0x7f2530107da0 2024-04-25T01:36:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.954+0000 7f2537034700 1 -- 172.21.15.78:0/2879244460 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253019c870 con 0x7f2530107da0 2024-04-25T01:36:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.954+0000 7f252dffb700 1 -- 172.21.15.78:0/2879244460 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f2524022020 con 0x7f2530107da0 2024-04-25T01:36:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.955+0000 7f252dffb700 1 --2- 172.21.15.78:0/2879244460 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f25200390a0 0x7f252003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.955+0000 7f2537034700 1 -- 172.21.15.78:0/2879244460 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2530063890 con 0x7f2530107da0 2024-04-25T01:36:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.955+0000 7f252dffb700 1 -- 172.21.15.78:0/2879244460 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f252404d840 con 0x7f2530107da0 2024-04-25T01:36:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.955+0000 7f252ffff700 1 -- 172.21.15.78:0/2879244460 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f25200390a0 msgr2=0x7f252003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1101495738 2024-04-25T01:36:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.955+0000 7f252ffff700 1 --2- 172.21.15.78:0/2879244460 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f25200390a0 0x7f252003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:10.960+0000 7f252dffb700 1 -- 172.21.15.78:0/2879244460 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f25240212c0 con 0x7f2530107da0 2024-04-25T01:36:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:11.156+0000 7f252ffff700 1 -- 172.21.15.78:0/2879244460 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f25200390a0 msgr2=0x7f252003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1101495738 2024-04-25T01:36:11.748 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:11.156+0000 7f252ffff700 1 --2- 172.21.15.78:0/2879244460 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f25200390a0 0x7f252003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:36:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:11.255+0000 7f2537034700 1 -- 172.21.15.78:0/2879244460 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f253019f2c0 con 0x7f2530107da0 2024-04-25T01:36:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:11.255+0000 7f252dffb700 1 -- 172.21.15.78:0/2879244460 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f252402d3f0 con 0x7f2530107da0 2024-04-25T01:36:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:11.258+0000 7f2537034700 1 -- 172.21.15.78:0/2879244460 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f25200390a0 msgr2=0x7f252003b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:36:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:11.258+0000 7f2537034700 1 --2- 172.21.15.78:0/2879244460 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f25200390a0 0x7f252003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:11.258+0000 7f2537034700 1 -- 172.21.15.78:0/2879244460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2530107da0 msgr2=0x7f253019bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:11.258+0000 7f2537034700 1 --2- 172.21.15.78:0/2879244460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2530107da0 0x7f253019bc40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25240117e0 tx=0x7f2524011ba0 comp rx=0 tx=0).stop 2024-04-25T01:36:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:11.258+0000 7f2537034700 1 -- 172.21.15.78:0/2879244460 shutdown_connections 2024-04-25T01:36:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:11.258+0000 7f2537034700 1 --2- 172.21.15.78:0/2879244460 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7f25200390a0 0x7f252003b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:11.258+0000 7f2537034700 1 --2- 172.21.15.78:0/2879244460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2530107da0 0x7f253019bc40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:11.258+0000 7f2537034700 1 -- 172.21.15.78:0/2879244460 >> 172.21.15.78:0/2879244460 conn(0x7f2530103360 msgr2=0x7f2530103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:11.258+0000 7f2537034700 1 -- 172.21.15.78:0/2879244460 shutdown_connections 2024-04-25T01:36:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:11.258+0000 7f2537034700 1 -- 172.21.15.78:0/2879244460 wait complete. 2024-04-25T01:36:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for the mgr to restart... 2024-04-25T01:36:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr epoch 9... 2024-04-25T01:36:12.130 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:11 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2879244460' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T01:36:12.131 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:12 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:12.021+0000 7f429184b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:36:12.491 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:12 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:12.199+0000 7f429184b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:36:12.491 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:12 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:12.370+0000 7f429184b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:36:12.933 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:12 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:12.661+0000 7f429184b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:36:12.934 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:12 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:12.754+0000 7f429184b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:36:12.934 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:12 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:12.958+0000 7f429184b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:36:13.241 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:13 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:13.151+0000 7f429184b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:36:13.991 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:13 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:13.746+0000 7f429184b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T01:36:14.447 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:14 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:14.170+0000 7f429184b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:36:14.447 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:14 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:14.267+0000 7f429184b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:36:14.447 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:14 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:14.470+0000 7f429184b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T01:36:15.241 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:14 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:14.997+0000 7f429184b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:36:15.241 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:15 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:15.090+0000 7f429184b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:36:15.659 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:15 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:15.334+0000 7f429184b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:36:15.659 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:15 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:15.469+0000 7f429184b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:36:15.659 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:15 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:15.565+0000 7f429184b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:36:15.660 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:15 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:15.683+0000 7f429184b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:36:15.991 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:15 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:15.776+0000 7f429184b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:36:16.839 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:16 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:16.589+0000 7f429184b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:36:16.839 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:36:16 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:36:16.809+0000 7f429184b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:36:17.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:16 smithi078 ceph-mon[29469]: Active manager daemon x restarted 2024-04-25T01:36:17.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:16 smithi078 ceph-mon[29469]: Activating manager daemon x 2024-04-25T01:36:17.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:16 smithi078 ceph-mon[29469]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T01:36:17.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:16 smithi078 ceph-mon[29469]: mgrmap e10: x(active, starting, since 0.00344505s) 2024-04-25T01:36:17.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:16 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:36:17.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:16 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T01:36:17.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:16 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:36:17.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:16 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:36:17.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:16 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:36:17.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:16 smithi078 ceph-mon[29469]: Manager daemon x is now available 2024-04-25T01:36:17.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:16 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:36:17.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:16 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:36:17.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T01:36:17.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:17 smithi078 ceph-mon[29469]: mgrmap e11: x(active, since 1.00621s) 2024-04-25T01:36:18.547 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:36:18.547 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T01:36:18.547 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T01:36:18.547 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:36:18.547 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.172+0000 7fa16b6cf700 1 Processor -- start 2024-04-25T01:36:18.547 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.172+0000 7fa16b6cf700 1 -- start start 2024-04-25T01:36:18.548 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.172+0000 7fa16b6cf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1641093d0 0x7fa1641097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:18.548 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.172+0000 7fa16b6cf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa164107960 con 0x7fa1641093d0 2024-04-25T01:36:18.548 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.172+0000 7fa16946b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1641093d0 0x7fa1641097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:18.548 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.173+0000 7fa16946b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1641093d0 0x7fa1641097f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56292/0 (socket says 172.21.15.78:56292) 2024-04-25T01:36:18.548 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.173+0000 7fa16946b700 1 -- 172.21.15.78:0/1798175081 learned_addr learned my addr 172.21.15.78:0/1798175081 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:18.548 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.173+0000 7fa16946b700 1 -- 172.21.15.78:0/1798175081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa164107ae0 con 0x7fa1641093d0 2024-04-25T01:36:18.548 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.173+0000 7fa16946b700 1 --2- 172.21.15.78:0/1798175081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1641093d0 0x7fa1641097f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa1600098a0 tx=0x7fa160009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5336eba32c19c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:18.548 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.173+0000 7fa15bfff700 1 -- 172.21.15.78:0/1798175081 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa160004160 con 0x7fa1641093d0 2024-04-25T01:36:18.548 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.173+0000 7fa15bfff700 1 -- 172.21.15.78:0/1798175081 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa160004300 con 0x7fa1641093d0 2024-04-25T01:36:18.548 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.173+0000 7fa15bfff700 1 -- 172.21.15.78:0/1798175081 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa160011b20 con 0x7fa1641093d0 2024-04-25T01:36:18.548 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.174+0000 7fa16b6cf700 1 -- 172.21.15.78:0/1798175081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1641093d0 msgr2=0x7fa1641097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:18.548 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.174+0000 7fa16b6cf700 1 --2- 172.21.15.78:0/1798175081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1641093d0 0x7fa1641097f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa1600098a0 tx=0x7fa160009bb0 comp rx=0 tx=0).stop 2024-04-25T01:36:18.548 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.174+0000 7fa16b6cf700 1 -- 172.21.15.78:0/1798175081 shutdown_connections 2024-04-25T01:36:18.549 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.174+0000 7fa16b6cf700 1 --2- 172.21.15.78:0/1798175081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1641093d0 0x7fa1641097f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:18.549 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.174+0000 7fa16b6cf700 1 -- 172.21.15.78:0/1798175081 >> 172.21.15.78:0/1798175081 conn(0x7fa1641032e0 msgr2=0x7fa164105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:18.549 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.174+0000 7fa16b6cf700 1 -- 172.21.15.78:0/1798175081 shutdown_connections 2024-04-25T01:36:18.549 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.174+0000 7fa16b6cf700 1 -- 172.21.15.78:0/1798175081 wait complete. 2024-04-25T01:36:18.549 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.175+0000 7fa16b6cf700 1 Processor -- start 2024-04-25T01:36:18.549 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.175+0000 7fa16b6cf700 1 -- start start 2024-04-25T01:36:18.549 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.175+0000 7fa16b6cf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1641093d0 0x7fa16407db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:18.549 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.175+0000 7fa16b6cf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa16410c2a0 con 0x7fa1641093d0 2024-04-25T01:36:18.549 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.175+0000 7fa16946b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1641093d0 0x7fa16407db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:18.549 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.175+0000 7fa16946b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1641093d0 0x7fa16407db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56308/0 (socket says 172.21.15.78:56308) 2024-04-25T01:36:18.549 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.175+0000 7fa16946b700 1 -- 172.21.15.78:0/4002073000 learned_addr learned my addr 172.21.15.78:0/4002073000 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:18.549 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.175+0000 7fa16946b700 1 -- 172.21.15.78:0/4002073000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16407e070 con 0x7fa1641093d0 2024-04-25T01:36:18.549 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.175+0000 7fa16946b700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1641093d0 0x7fa16407db30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa1600117e0 tx=0x7fa160011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:18.549 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.176+0000 7fa15a7fc700 1 -- 172.21.15.78:0/4002073000 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa160003bf0 con 0x7fa1641093d0 2024-04-25T01:36:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.176+0000 7fa15a7fc700 1 -- 172.21.15.78:0/4002073000 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa16001a910 con 0x7fa1641093d0 2024-04-25T01:36:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.176+0000 7fa15a7fc700 1 -- 172.21.15.78:0/4002073000 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa160024970 con 0x7fa1641093d0 2024-04-25T01:36:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.176+0000 7fa16b6cf700 1 -- 172.21.15.78:0/4002073000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16407c260 con 0x7fa1641093d0 2024-04-25T01:36:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.176+0000 7fa16b6cf700 1 -- 172.21.15.78:0/4002073000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa16407c720 con 0x7fa1641093d0 2024-04-25T01:36:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.177+0000 7fa15a7fc700 1 -- 172.21.15.78:0/4002073000 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fa160022020 con 0x7fa1641093d0 2024-04-25T01:36:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.177+0000 7fa15a7fc700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fa1500390a0 0x7fa15003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.177+0000 7fa15a7fc700 1 -- 172.21.15.78:0/4002073000 --> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa15003bc70 con 0x7fa1500390a0 2024-04-25T01:36:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.177+0000 7fa15a7fc700 1 -- 172.21.15.78:0/4002073000 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa16004d990 con 0x7fa1641093d0 2024-04-25T01:36:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.177+0000 7fa168c6a700 1 -- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fa1500390a0 msgr2=0x7fa15003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1101495738 2024-04-25T01:36:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.177+0000 7fa168c6a700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fa1500390a0 0x7fa15003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:36:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.378+0000 7fa168c6a700 1 -- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fa1500390a0 msgr2=0x7fa15003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1101495738 2024-04-25T01:36:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.378+0000 7fa168c6a700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fa1500390a0 0x7fa15003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:36:18.550 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.779+0000 7fa168c6a700 1 -- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fa1500390a0 msgr2=0x7fa15003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1101495738 2024-04-25T01:36:18.551 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:12.779+0000 7fa168c6a700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fa1500390a0 0x7fa15003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:36:18.551 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:13.579+0000 7fa168c6a700 1 -- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fa1500390a0 msgr2=0x7fa15003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1101495738 2024-04-25T01:36:18.551 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:13.579+0000 7fa168c6a700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fa1500390a0 0x7fa15003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:36:18.551 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:15.180+0000 7fa168c6a700 1 -- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fa1500390a0 msgr2=0x7fa15003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1101495738 2024-04-25T01:36:18.551 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:15.180+0000 7fa168c6a700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fa1500390a0 0x7fa15003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T01:36:18.551 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:16.813+0000 7fa15a7fc700 1 -- 172.21.15.78:0/4002073000 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fa16001ab30 con 0x7fa1641093d0 2024-04-25T01:36:18.551 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:16.813+0000 7fa15a7fc700 1 -- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fa1500390a0 msgr2=0x7fa15003b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:36:18.551 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:16.813+0000 7fa15a7fc700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fa1500390a0 0x7fa15003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:18.551 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.815+0000 7fa15a7fc700 1 -- 172.21.15.78:0/4002073000 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fa16004fb10 con 0x7fa1641093d0 2024-04-25T01:36:18.551 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.815+0000 7fa15a7fc700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa15003cb60 0x7fa15003ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:18.551 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.815+0000 7fa15a7fc700 1 -- 172.21.15.78:0/4002073000 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa160027e50 con 0x7fa15003cb60 2024-04-25T01:36:18.551 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.815+0000 7fa168c6a700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa15003cb60 0x7fa15003ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:18.551 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.816+0000 7fa168c6a700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa15003cb60 0x7fa15003ef50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa154003a10 tx=0x7fa1540092b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:18.551 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.817+0000 7fa15a7fc700 1 -- 172.21.15.78:0/4002073000 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fa160027e50 con 0x7fa15003cb60 2024-04-25T01:36:18.552 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.823+0000 7fa16b6cf700 1 -- 172.21.15.78:0/4002073000 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa16410c0f0 con 0x7fa15003cb60 2024-04-25T01:36:18.552 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.824+0000 7fa15a7fc700 1 -- 172.21.15.78:0/4002073000 <== mgr.14148 v2:172.21.15.78:6800/2190983290 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fa16410c0f0 con 0x7fa15003cb60 2024-04-25T01:36:18.552 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.824+0000 7fa16b6cf700 1 -- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa15003cb60 msgr2=0x7fa15003ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:18.552 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.824+0000 7fa16b6cf700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa15003cb60 0x7fa15003ef50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa154003a10 tx=0x7fa1540092b0 comp rx=0 tx=0).stop 2024-04-25T01:36:18.552 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.824+0000 7fa16b6cf700 1 -- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1641093d0 msgr2=0x7fa16407db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:18.552 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.824+0000 7fa16b6cf700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1641093d0 0x7fa16407db30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa1600117e0 tx=0x7fa160011ba0 comp rx=0 tx=0).stop 2024-04-25T01:36:18.552 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.824+0000 7fa16b6cf700 1 -- 172.21.15.78:0/4002073000 shutdown_connections 2024-04-25T01:36:18.552 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.824+0000 7fa16b6cf700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa15003cb60 0x7fa15003ef50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:18.552 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.824+0000 7fa16b6cf700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1641093d0 0x7fa16407db30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:18.552 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.824+0000 7fa16b6cf700 1 --2- 172.21.15.78:0/4002073000 >> [v2:172.21.15.78:6800/1101495738,v1:172.21.15.78:6801/1101495738] conn(0x7fa1500390a0 0x7fa15003b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:18.552 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.824+0000 7fa16b6cf700 1 -- 172.21.15.78:0/4002073000 >> 172.21.15.78:0/4002073000 conn(0x7fa1641032e0 msgr2=0x7fa164103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:18.552 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.824+0000 7fa16b6cf700 1 -- 172.21.15.78:0/4002073000 shutdown_connections 2024-04-25T01:36:18.552 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:17.824+0000 7fa16b6cf700 1 -- 172.21.15.78:0/4002073000 wait complete. 2024-04-25T01:36:18.552 INFO:teuthology.orchestra.run.smithi078.stdout:mgr epoch 9 is available 2024-04-25T01:36:18.553 INFO:teuthology.orchestra.run.smithi078.stdout:Generating a dashboard self-signed certificate... 2024-04-25T01:36:18.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:18 smithi078 ceph-mon[29469]: [25/Apr/2024:01:36:17] ENGINE Bus STARTING 2024-04-25T01:36:18.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:18 smithi078 ceph-mon[29469]: [25/Apr/2024:01:36:17] ENGINE Serving on http://172.21.15.78:8765 2024-04-25T01:36:18.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:18 smithi078 ceph-mon[29469]: [25/Apr/2024:01:36:17] ENGINE Serving on https://172.21.15.78:7150 2024-04-25T01:36:18.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:18 smithi078 ceph-mon[29469]: [25/Apr/2024:01:36:17] ENGINE Bus STARTED 2024-04-25T01:36:18.956 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:18 smithi078 ceph-mon[29469]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T01:36:18.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:18 smithi078 ceph-mon[29469]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T01:36:19.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T01:36:19.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.009+0000 7f89a6676700 1 Processor -- start 2024-04-25T01:36:19.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.009+0000 7f89a6676700 1 -- start start 2024-04-25T01:36:19.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.009+0000 7f89a6676700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a007e6a0 0x7f89a007cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:19.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.009+0000 7f89a6676700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89a007d120 con 0x7f89a007e6a0 2024-04-25T01:36:19.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.009+0000 7f899ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a007e6a0 0x7f89a007cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:19.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.009+0000 7f899ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a007e6a0 0x7f89a007cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56384/0 (socket says 172.21.15.78:56384) 2024-04-25T01:36:19.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.009+0000 7f899ffff700 1 -- 172.21.15.78:0/3813086722 learned_addr learned my addr 172.21.15.78:0/3813086722 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:19.817 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.009+0000 7f899ffff700 1 -- 172.21.15.78:0/3813086722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a007d2a0 con 0x7f89a007e6a0 2024-04-25T01:36:19.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.009+0000 7f899ffff700 1 --2- 172.21.15.78:0/3813086722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a007e6a0 0x7f89a007cb50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89900098a0 tx=0x7f8990009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0e7b1e7b35d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:19.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.010+0000 7f899effd700 1 -- 172.21.15.78:0/3813086722 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8990004160 con 0x7f89a007e6a0 2024-04-25T01:36:19.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.010+0000 7f899effd700 1 -- 172.21.15.78:0/3813086722 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8990004300 con 0x7f89a007e6a0 2024-04-25T01:36:19.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.010+0000 7f899effd700 1 -- 172.21.15.78:0/3813086722 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8990011b20 con 0x7f89a007e6a0 2024-04-25T01:36:19.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.010+0000 7f89a6676700 1 -- 172.21.15.78:0/3813086722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a007e6a0 msgr2=0x7f89a007cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:19.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.010+0000 7f89a6676700 1 --2- 172.21.15.78:0/3813086722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a007e6a0 0x7f89a007cb50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89900098a0 tx=0x7f8990009bb0 comp rx=0 tx=0).stop 2024-04-25T01:36:19.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.010+0000 7f89a6676700 1 -- 172.21.15.78:0/3813086722 shutdown_connections 2024-04-25T01:36:19.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.010+0000 7f89a6676700 1 --2- 172.21.15.78:0/3813086722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a007e6a0 0x7f89a007cb50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:19.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.010+0000 7f89a6676700 1 -- 172.21.15.78:0/3813086722 >> 172.21.15.78:0/3813086722 conn(0x7f89a0103280 msgr2=0x7f89a0105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:19.818 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.010+0000 7f89a6676700 1 -- 172.21.15.78:0/3813086722 shutdown_connections 2024-04-25T01:36:19.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.010+0000 7f89a6676700 1 -- 172.21.15.78:0/3813086722 wait complete. 2024-04-25T01:36:19.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.011+0000 7f89a6676700 1 Processor -- start 2024-04-25T01:36:19.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.011+0000 7f89a6676700 1 -- start start 2024-04-25T01:36:19.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.011+0000 7f89a6676700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a007e6a0 0x7f89a019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:19.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.011+0000 7f89a6676700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89a010e320 con 0x7f89a007e6a0 2024-04-25T01:36:19.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.011+0000 7f899ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a007e6a0 0x7f89a019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:19.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.011+0000 7f899ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a007e6a0 0x7f89a019ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56396/0 (socket says 172.21.15.78:56396) 2024-04-25T01:36:19.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.011+0000 7f899ffff700 1 -- 172.21.15.78:0/214169915 learned_addr learned my addr 172.21.15.78:0/214169915 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:19.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.012+0000 7f899ffff700 1 -- 172.21.15.78:0/214169915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a019e310 con 0x7f89a007e6a0 2024-04-25T01:36:19.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.012+0000 7f899ffff700 1 --2- 172.21.15.78:0/214169915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a007e6a0 0x7f89a019ddd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f89900117e0 tx=0x7f8990011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:19.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.012+0000 7f899d7fa700 1 -- 172.21.15.78:0/214169915 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8990003bf0 con 0x7f89a007e6a0 2024-04-25T01:36:19.819 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.012+0000 7f89a6676700 1 -- 172.21.15.78:0/214169915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a019e5a0 con 0x7f89a007e6a0 2024-04-25T01:36:19.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.012+0000 7f899d7fa700 1 -- 172.21.15.78:0/214169915 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f899001a910 con 0x7f89a007e6a0 2024-04-25T01:36:19.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.012+0000 7f899d7fa700 1 -- 172.21.15.78:0/214169915 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8990024970 con 0x7f89a007e6a0 2024-04-25T01:36:19.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.012+0000 7f89a6676700 1 -- 172.21.15.78:0/214169915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a019ea00 con 0x7f89a007e6a0 2024-04-25T01:36:19.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.013+0000 7f899d7fa700 1 -- 172.21.15.78:0/214169915 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f8990022020 con 0x7f89a007e6a0 2024-04-25T01:36:19.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.013+0000 7f89a6676700 1 -- 172.21.15.78:0/214169915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a0063890 con 0x7f89a007e6a0 2024-04-25T01:36:19.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.013+0000 7f899d7fa700 1 --2- 172.21.15.78:0/214169915 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8988039050 0x7f898803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:19.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.013+0000 7f899d7fa700 1 -- 172.21.15.78:0/214169915 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f899004d520 con 0x7f89a007e6a0 2024-04-25T01:36:19.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.013+0000 7f899f7fe700 1 --2- 172.21.15.78:0/214169915 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8988039050 0x7f898803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:19.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.013+0000 7f899f7fe700 1 --2- 172.21.15.78:0/214169915 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8988039050 0x7f898803b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8994006fd0 tx=0x7f8994006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:19.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.018+0000 7f899d7fa700 1 -- 172.21.15.78:0/214169915 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8990011e20 con 0x7f89a007e6a0 2024-04-25T01:36:19.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.229+0000 7f89a6676700 1 -- 172.21.15.78:0/214169915 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f89a0104450 con 0x7f8988039050 2024-04-25T01:36:19.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.317+0000 7f899d7fa700 1 -- 172.21.15.78:0/214169915 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f89a0104450 con 0x7f8988039050 2024-04-25T01:36:19.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.319+0000 7f89a6676700 1 -- 172.21.15.78:0/214169915 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8988039050 msgr2=0x7f898803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:19.820 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.319+0000 7f89a6676700 1 --2- 172.21.15.78:0/214169915 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8988039050 0x7f898803b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8994006fd0 tx=0x7f8994006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:19.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.319+0000 7f89a6676700 1 -- 172.21.15.78:0/214169915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a007e6a0 msgr2=0x7f89a019ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:19.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.319+0000 7f89a6676700 1 --2- 172.21.15.78:0/214169915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a007e6a0 0x7f89a019ddd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f89900117e0 tx=0x7f8990011c10 comp rx=0 tx=0).stop 2024-04-25T01:36:19.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.319+0000 7f89a6676700 1 -- 172.21.15.78:0/214169915 shutdown_connections 2024-04-25T01:36:19.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.319+0000 7f89a6676700 1 --2- 172.21.15.78:0/214169915 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8988039050 0x7f898803b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:19.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.319+0000 7f89a6676700 1 --2- 172.21.15.78:0/214169915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a007e6a0 0x7f89a019ddd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:19.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.319+0000 7f89a6676700 1 -- 172.21.15.78:0/214169915 >> 172.21.15.78:0/214169915 conn(0x7f89a0103280 msgr2=0x7f89a0103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:19.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.320+0000 7f89a6676700 1 -- 172.21.15.78:0/214169915 shutdown_connections 2024-04-25T01:36:19.821 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:19.320+0000 7f89a6676700 1 -- 172.21.15.78:0/214169915 wait complete. 2024-04-25T01:36:19.821 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial admin user... 2024-04-25T01:36:20.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:19 smithi078 ceph-mon[29469]: mgrmap e12: x(active, since 2s) 2024-04-25T01:36:20.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:20.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:20.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:21.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:20 smithi078 ceph-mon[29469]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:21.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:20 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:21.217 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:20 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:21.224 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$hsjpIDcYWfNketZ6WMmRze2eUeASWE3FC7ZV3O5jv2d2ljx3jAuZq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714008980, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T01:36:21.224 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.309+0000 7ff2bc43c700 1 Processor -- start 2024-04-25T01:36:21.224 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.309+0000 7ff2bc43c700 1 -- start start 2024-04-25T01:36:21.224 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.309+0000 7ff2bc43c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4105fd0 0x7ff2b41063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:21.224 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.309+0000 7ff2bc43c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41069c0 con 0x7ff2b4105fd0 2024-04-25T01:36:21.225 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.309+0000 7ff2ba1d8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4105fd0 0x7ff2b41063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:21.225 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.309+0000 7ff2ba1d8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4105fd0 0x7ff2b41063f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53526/0 (socket says 172.21.15.78:53526) 2024-04-25T01:36:21.225 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.309+0000 7ff2ba1d8700 1 -- 172.21.15.78:0/2049279864 learned_addr learned my addr 172.21.15.78:0/2049279864 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:21.225 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.310+0000 7ff2ba1d8700 1 -- 172.21.15.78:0/2049279864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b4108f60 con 0x7ff2b4105fd0 2024-04-25T01:36:21.225 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.310+0000 7ff2ba1d8700 1 --2- 172.21.15.78:0/2049279864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4105fd0 0x7ff2b41063f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00098a0 tx=0x7ff2b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303070d337bd4ab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:21.225 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.310+0000 7ff2b91d6700 1 -- 172.21.15.78:0/2049279864 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0004160 con 0x7ff2b4105fd0 2024-04-25T01:36:21.225 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.310+0000 7ff2b91d6700 1 -- 172.21.15.78:0/2049279864 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b0004300 con 0x7ff2b4105fd0 2024-04-25T01:36:21.225 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.310+0000 7ff2b91d6700 1 -- 172.21.15.78:0/2049279864 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0011b20 con 0x7ff2b4105fd0 2024-04-25T01:36:21.225 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.310+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2049279864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4105fd0 msgr2=0x7ff2b41063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:21.225 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.310+0000 7ff2bc43c700 1 --2- 172.21.15.78:0/2049279864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4105fd0 0x7ff2b41063f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00098a0 tx=0x7ff2b0009bb0 comp rx=0 tx=0).stop 2024-04-25T01:36:21.225 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.311+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2049279864 shutdown_connections 2024-04-25T01:36:21.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.311+0000 7ff2bc43c700 1 --2- 172.21.15.78:0/2049279864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4105fd0 0x7ff2b41063f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:21.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.311+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2049279864 >> 172.21.15.78:0/2049279864 conn(0x7ff2b4101900 msgr2=0x7ff2b4103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:21.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.311+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2049279864 shutdown_connections 2024-04-25T01:36:21.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.311+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2049279864 wait complete. 2024-04-25T01:36:21.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.311+0000 7ff2bc43c700 1 Processor -- start 2024-04-25T01:36:21.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.311+0000 7ff2bc43c700 1 -- start start 2024-04-25T01:36:21.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.311+0000 7ff2bc43c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4105fd0 0x7ff2b419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:21.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.311+0000 7ff2bc43c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b410c310 con 0x7ff2b4105fd0 2024-04-25T01:36:21.226 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.311+0000 7ff2ba1d8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4105fd0 0x7ff2b419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:21.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.311+0000 7ff2ba1d8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4105fd0 0x7ff2b419bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53530/0 (socket says 172.21.15.78:53530) 2024-04-25T01:36:21.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.311+0000 7ff2ba1d8700 1 -- 172.21.15.78:0/2994243398 learned_addr learned my addr 172.21.15.78:0/2994243398 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:21.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.312+0000 7ff2ba1d8700 1 -- 172.21.15.78:0/2994243398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b419c190 con 0x7ff2b4105fd0 2024-04-25T01:36:21.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.312+0000 7ff2ba1d8700 1 --2- 172.21.15.78:0/2994243398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4105fd0 0x7ff2b419bc50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00117e0 tx=0x7ff2b0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:21.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.312+0000 7ff2ab7fe700 1 -- 172.21.15.78:0/2994243398 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0003bf0 con 0x7ff2b4105fd0 2024-04-25T01:36:21.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.312+0000 7ff2ab7fe700 1 -- 172.21.15.78:0/2994243398 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b001a910 con 0x7ff2b4105fd0 2024-04-25T01:36:21.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.312+0000 7ff2ab7fe700 1 -- 172.21.15.78:0/2994243398 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0024970 con 0x7ff2b4105fd0 2024-04-25T01:36:21.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.312+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2994243398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b419c420 con 0x7ff2b4105fd0 2024-04-25T01:36:21.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.312+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2994243398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b419c880 con 0x7ff2b4105fd0 2024-04-25T01:36:21.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.313+0000 7ff2ab7fe700 1 -- 172.21.15.78:0/2994243398 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ff2b0022020 con 0x7ff2b4105fd0 2024-04-25T01:36:21.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.313+0000 7ff2ab7fe700 1 --2- 172.21.15.78:0/2994243398 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff2a00390a0 0x7ff2a003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:21.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.313+0000 7ff2ab7fe700 1 -- 172.21.15.78:0/2994243398 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff2b004d910 con 0x7ff2b4105fd0 2024-04-25T01:36:21.227 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.313+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2994243398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b4063890 con 0x7ff2b4105fd0 2024-04-25T01:36:21.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.313+0000 7ff2b99d7700 1 --2- 172.21.15.78:0/2994243398 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff2a00390a0 0x7ff2a003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:21.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.314+0000 7ff2b99d7700 1 --2- 172.21.15.78:0/2994243398 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff2a00390a0 0x7ff2a003b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4006fd0 tx=0x7ff2a4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:21.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.318+0000 7ff2ab7fe700 1 -- 172.21.15.78:0/2994243398 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff2b002d3f0 con 0x7ff2b4105fd0 2024-04-25T01:36:21.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.536+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2994243398 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7ff2b419f1d0 con 0x7ff2a00390a0 2024-04-25T01:36:21.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.764+0000 7ff2ab7fe700 1 -- 172.21.15.78:0/2994243398 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7ff2b419f1d0 con 0x7ff2a00390a0 2024-04-25T01:36:21.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.767+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2994243398 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff2a00390a0 msgr2=0x7ff2a003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:21.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.767+0000 7ff2bc43c700 1 --2- 172.21.15.78:0/2994243398 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff2a00390a0 0x7ff2a003b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4006fd0 tx=0x7ff2a4006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:21.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.767+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2994243398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4105fd0 msgr2=0x7ff2b419bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:21.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.767+0000 7ff2bc43c700 1 --2- 172.21.15.78:0/2994243398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4105fd0 0x7ff2b419bc50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00117e0 tx=0x7ff2b0011ba0 comp rx=0 tx=0).stop 2024-04-25T01:36:21.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.767+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2994243398 shutdown_connections 2024-04-25T01:36:21.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.767+0000 7ff2bc43c700 1 --2- 172.21.15.78:0/2994243398 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff2a00390a0 0x7ff2a003b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:21.228 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.767+0000 7ff2bc43c700 1 --2- 172.21.15.78:0/2994243398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2b4105fd0 0x7ff2b419bc50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:21.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.767+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2994243398 >> 172.21.15.78:0/2994243398 conn(0x7ff2b4101900 msgr2=0x7ff2b41023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:21.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.767+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2994243398 shutdown_connections 2024-04-25T01:36:21.229 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:20.767+0000 7ff2bc43c700 1 -- 172.21.15.78:0/2994243398 wait complete. 2024-04-25T01:36:21.229 INFO:teuthology.orchestra.run.smithi078.stdout:Fetching dashboard port number... 2024-04-25T01:36:22.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:21 smithi078 ceph-mon[29469]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:22.491 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T01:36:22.492 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.657+0000 7f13fa7ae700 1 Processor -- start 2024-04-25T01:36:22.492 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.658+0000 7f13fa7ae700 1 -- start start 2024-04-25T01:36:22.492 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.658+0000 7f13fa7ae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4107d60 0x7f13f4108180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:22.492 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.658+0000 7f13fa7ae700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4108750 con 0x7f13f4107d60 2024-04-25T01:36:22.492 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.658+0000 7f13f3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4107d60 0x7f13f4108180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:22.492 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.658+0000 7f13f3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4107d60 0x7f13f4108180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53538/0 (socket says 172.21.15.78:53538) 2024-04-25T01:36:22.492 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.658+0000 7f13f3fff700 1 -- 172.21.15.78:0/4175789645 learned_addr learned my addr 172.21.15.78:0/4175789645 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:22.492 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.658+0000 7f13f3fff700 1 -- 172.21.15.78:0/4175789645 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f4108f10 con 0x7f13f4107d60 2024-04-25T01:36:22.492 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.658+0000 7f13f3fff700 1 --2- 172.21.15.78:0/4175789645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4107d60 0x7f13f4108180 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f13e80098a0 tx=0x7f13e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8da632e0fe3efb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:22.492 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.658+0000 7f13f2ffd700 1 -- 172.21.15.78:0/4175789645 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e8004160 con 0x7f13f4107d60 2024-04-25T01:36:22.492 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.659+0000 7f13f2ffd700 1 -- 172.21.15.78:0/4175789645 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e8004300 con 0x7f13f4107d60 2024-04-25T01:36:22.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.659+0000 7f13f2ffd700 1 -- 172.21.15.78:0/4175789645 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e8011b20 con 0x7f13f4107d60 2024-04-25T01:36:22.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.659+0000 7f13fa7ae700 1 -- 172.21.15.78:0/4175789645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4107d60 msgr2=0x7f13f4108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:22.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.659+0000 7f13fa7ae700 1 --2- 172.21.15.78:0/4175789645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4107d60 0x7f13f4108180 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f13e80098a0 tx=0x7f13e8009bb0 comp rx=0 tx=0).stop 2024-04-25T01:36:22.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.659+0000 7f13fa7ae700 1 -- 172.21.15.78:0/4175789645 shutdown_connections 2024-04-25T01:36:22.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.659+0000 7f13fa7ae700 1 --2- 172.21.15.78:0/4175789645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4107d60 0x7f13f4108180 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:22.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.659+0000 7f13fa7ae700 1 -- 172.21.15.78:0/4175789645 >> 172.21.15.78:0/4175789645 conn(0x7f13f4103380 msgr2=0x7f13f41057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:22.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.659+0000 7f13fa7ae700 1 -- 172.21.15.78:0/4175789645 shutdown_connections 2024-04-25T01:36:22.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.659+0000 7f13fa7ae700 1 -- 172.21.15.78:0/4175789645 wait complete. 2024-04-25T01:36:22.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.660+0000 7f13fa7ae700 1 Processor -- start 2024-04-25T01:36:22.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.660+0000 7f13fa7ae700 1 -- start start 2024-04-25T01:36:22.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.660+0000 7f13fa7ae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4107d60 0x7f13f419bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:22.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.660+0000 7f13fa7ae700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13f410c2c0 con 0x7f13f4107d60 2024-04-25T01:36:22.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.660+0000 7f13f3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4107d60 0x7f13f419bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:22.493 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.660+0000 7f13f3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4107d60 0x7f13f419bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53548/0 (socket says 172.21.15.78:53548) 2024-04-25T01:36:22.494 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.660+0000 7f13f3fff700 1 -- 172.21.15.78:0/3950663750 learned_addr learned my addr 172.21.15.78:0/3950663750 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:22.494 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.660+0000 7f13f3fff700 1 -- 172.21.15.78:0/3950663750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f419c160 con 0x7f13f4107d60 2024-04-25T01:36:22.494 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.661+0000 7f13f3fff700 1 --2- 172.21.15.78:0/3950663750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4107d60 0x7f13f419bc20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f13e80117e0 tx=0x7f13e8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:22.494 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.661+0000 7f13f17fa700 1 -- 172.21.15.78:0/3950663750 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e8003bf0 con 0x7f13f4107d60 2024-04-25T01:36:22.494 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.661+0000 7f13f17fa700 1 -- 172.21.15.78:0/3950663750 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e801a910 con 0x7f13f4107d60 2024-04-25T01:36:22.494 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.661+0000 7f13fa7ae700 1 -- 172.21.15.78:0/3950663750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f419c3f0 con 0x7f13f4107d60 2024-04-25T01:36:22.494 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.661+0000 7f13f17fa700 1 -- 172.21.15.78:0/3950663750 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e80249a0 con 0x7f13f4107d60 2024-04-25T01:36:22.494 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.661+0000 7f13fa7ae700 1 -- 172.21.15.78:0/3950663750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f419c850 con 0x7f13f4107d60 2024-04-25T01:36:22.494 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.662+0000 7f13f17fa700 1 -- 172.21.15.78:0/3950663750 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f13e8022020 con 0x7f13f4107d60 2024-04-25T01:36:22.494 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.662+0000 7f13fa7ae700 1 -- 172.21.15.78:0/3950663750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f4063890 con 0x7f13f4107d60 2024-04-25T01:36:22.494 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.662+0000 7f13f17fa700 1 --2- 172.21.15.78:0/3950663750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f13dc0390b0 0x7f13dc03b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:22.494 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.662+0000 7f13f17fa700 1 -- 172.21.15.78:0/3950663750 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f13e804d5f0 con 0x7f13f4107d60 2024-04-25T01:36:22.494 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.662+0000 7f13f37fe700 1 --2- 172.21.15.78:0/3950663750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f13dc0390b0 0x7f13dc03b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:22.495 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.662+0000 7f13f37fe700 1 --2- 172.21.15.78:0/3950663750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f13dc0390b0 0x7f13dc03b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f13e4006fd0 tx=0x7f13e4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:22.495 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.667+0000 7f13f17fa700 1 -- 172.21.15.78:0/3950663750 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f13e8011e20 con 0x7f13f4107d60 2024-04-25T01:36:22.495 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.899+0000 7f13fa7ae700 1 -- 172.21.15.78:0/3950663750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f13f419f220 con 0x7f13f4107d60 2024-04-25T01:36:22.495 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.899+0000 7f13f17fa700 1 -- 172.21.15.78:0/3950663750 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f13e800ba00 con 0x7f13f4107d60 2024-04-25T01:36:22.495 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.901+0000 7f13fa7ae700 1 -- 172.21.15.78:0/3950663750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f13dc0390b0 msgr2=0x7f13dc03b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:22.495 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.901+0000 7f13fa7ae700 1 --2- 172.21.15.78:0/3950663750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f13dc0390b0 0x7f13dc03b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f13e4006fd0 tx=0x7f13e4006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:22.495 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.901+0000 7f13fa7ae700 1 -- 172.21.15.78:0/3950663750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4107d60 msgr2=0x7f13f419bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:22.495 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.901+0000 7f13fa7ae700 1 --2- 172.21.15.78:0/3950663750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4107d60 0x7f13f419bc20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f13e80117e0 tx=0x7f13e8011c10 comp rx=0 tx=0).stop 2024-04-25T01:36:22.495 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.901+0000 7f13fa7ae700 1 -- 172.21.15.78:0/3950663750 shutdown_connections 2024-04-25T01:36:22.495 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.901+0000 7f13fa7ae700 1 --2- 172.21.15.78:0/3950663750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f13dc0390b0 0x7f13dc03b570 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:22.495 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.901+0000 7f13fa7ae700 1 --2- 172.21.15.78:0/3950663750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13f4107d60 0x7f13f419bc20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:22.495 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.901+0000 7f13fa7ae700 1 -- 172.21.15.78:0/3950663750 >> 172.21.15.78:0/3950663750 conn(0x7f13f4103380 msgr2=0x7f13f4103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:22.495 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.901+0000 7f13fa7ae700 1 -- 172.21.15.78:0/3950663750 shutdown_connections 2024-04-25T01:36:22.495 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:21.901+0000 7f13fa7ae700 1 -- 172.21.15.78:0/3950663750 wait complete. 2024-04-25T01:36:22.497 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T01:36:22.498 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-04-25T01:36:22.504 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T01:36:22.505 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-04-25T01:36:22.505 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-04-25T01:36:22.505 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T01:36:22.506 INFO:teuthology.orchestra.run.smithi078.stdout:Ceph Dashboard is now available at: 2024-04-25T01:36:22.506 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:36:22.506 INFO:teuthology.orchestra.run.smithi078.stdout: URL: https://smithi078:8443/ 2024-04-25T01:36:22.506 INFO:teuthology.orchestra.run.smithi078.stdout: User: admin 2024-04-25T01:36:22.506 INFO:teuthology.orchestra.run.smithi078.stdout: Password: 2g5hxan9mb 2024-04-25T01:36:22.507 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:36:22.507 INFO:teuthology.orchestra.run.smithi078.stdout:Saving cluster configuration to /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config directory 2024-04-25T01:36:22.507 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling autotune for osd_memory_target 2024-04-25T01:36:23.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:22 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3950663750' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T01:36:23.952 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:23 smithi078 ceph-mon[29469]: mgrmap e13: x(active, since 6s) 2024-04-25T01:36:24.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.772+0000 7fe2ba48c700 1 Processor -- start 2024-04-25T01:36:24.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.772+0000 7fe2ba48c700 1 -- start start 2024-04-25T01:36:24.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.772+0000 7fe2ba48c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2b407e650 0x7fe2b407cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:24.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.772+0000 7fe2ba48c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b407d090 con 0x7fe2b407e650 2024-04-25T01:36:24.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.773+0000 7fe2b3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2b407e650 0x7fe2b407cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:24.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.773+0000 7fe2b3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2b407e650 0x7fe2b407cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53550/0 (socket says 172.21.15.78:53550) 2024-04-25T01:36:24.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.773+0000 7fe2b3fff700 1 -- 172.21.15.78:0/530794470 learned_addr learned my addr 172.21.15.78:0/530794470 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:24.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.773+0000 7fe2b3fff700 1 -- 172.21.15.78:0/530794470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b407d210 con 0x7fe2b407e650 2024-04-25T01:36:24.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.773+0000 7fe2b3fff700 1 --2- 172.21.15.78:0/530794470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2b407e650 0x7fe2b407cb50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40098a0 tx=0x7fe2a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92943e3d3ab5d82d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.773+0000 7fe2b2ffd700 1 -- 172.21.15.78:0/530794470 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2a4004160 con 0x7fe2b407e650 2024-04-25T01:36:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.773+0000 7fe2b2ffd700 1 -- 172.21.15.78:0/530794470 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2a4004300 con 0x7fe2b407e650 2024-04-25T01:36:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.773+0000 7fe2b2ffd700 1 -- 172.21.15.78:0/530794470 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2a4011b20 con 0x7fe2b407e650 2024-04-25T01:36:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.774+0000 7fe2ba48c700 1 -- 172.21.15.78:0/530794470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2b407e650 msgr2=0x7fe2b407cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.774+0000 7fe2ba48c700 1 --2- 172.21.15.78:0/530794470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2b407e650 0x7fe2b407cb50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40098a0 tx=0x7fe2a4009bb0 comp rx=0 tx=0).stop 2024-04-25T01:36:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.774+0000 7fe2ba48c700 1 -- 172.21.15.78:0/530794470 shutdown_connections 2024-04-25T01:36:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.774+0000 7fe2ba48c700 1 --2- 172.21.15.78:0/530794470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2b407e650 0x7fe2b407cb50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.774+0000 7fe2ba48c700 1 -- 172.21.15.78:0/530794470 >> 172.21.15.78:0/530794470 conn(0x7fe2b4103380 msgr2=0x7fe2b41057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.774+0000 7fe2ba48c700 1 -- 172.21.15.78:0/530794470 shutdown_connections 2024-04-25T01:36:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.774+0000 7fe2ba48c700 1 -- 172.21.15.78:0/530794470 wait complete. 2024-04-25T01:36:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.774+0000 7fe2ba48c700 1 Processor -- start 2024-04-25T01:36:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.775+0000 7fe2ba48c700 1 -- start start 2024-04-25T01:36:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.775+0000 7fe2ba48c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2b407e650 0x7fe2b419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.775+0000 7fe2ba48c700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b410e320 con 0x7fe2b407e650 2024-04-25T01:36:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.775+0000 7fe2b3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2b407e650 0x7fe2b419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.775+0000 7fe2b3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2b407e650 0x7fe2b419ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53560/0 (socket says 172.21.15.78:53560) 2024-04-25T01:36:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.775+0000 7fe2b3fff700 1 -- 172.21.15.78:0/2336213452 learned_addr learned my addr 172.21.15.78:0/2336213452 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.775+0000 7fe2b3fff700 1 -- 172.21.15.78:0/2336213452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b419e310 con 0x7fe2b407e650 2024-04-25T01:36:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.775+0000 7fe2b3fff700 1 --2- 172.21.15.78:0/2336213452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2b407e650 0x7fe2b419ddd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40117e0 tx=0x7fe2a4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.776+0000 7fe2b17fa700 1 -- 172.21.15.78:0/2336213452 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2a4003bf0 con 0x7fe2b407e650 2024-04-25T01:36:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.776+0000 7fe2b17fa700 1 -- 172.21.15.78:0/2336213452 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2a401a910 con 0x7fe2b407e650 2024-04-25T01:36:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.776+0000 7fe2ba48c700 1 -- 172.21.15.78:0/2336213452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b419e5a0 con 0x7fe2b407e650 2024-04-25T01:36:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.776+0000 7fe2b17fa700 1 -- 172.21.15.78:0/2336213452 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2a4024970 con 0x7fe2b407e650 2024-04-25T01:36:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.776+0000 7fe2ba48c700 1 -- 172.21.15.78:0/2336213452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b419ea00 con 0x7fe2b407e650 2024-04-25T01:36:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.776+0000 7fe2b17fa700 1 -- 172.21.15.78:0/2336213452 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe2a4022020 con 0x7fe2b407e650 2024-04-25T01:36:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.777+0000 7fe2ba48c700 1 -- 172.21.15.78:0/2336213452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b4063890 con 0x7fe2b407e650 2024-04-25T01:36:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.777+0000 7fe2b17fa700 1 --2- 172.21.15.78:0/2336213452 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe29c0390f0 0x7fe29c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:24.503 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.777+0000 7fe2b17fa700 1 -- 172.21.15.78:0/2336213452 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe2a404d770 con 0x7fe2b407e650 2024-04-25T01:36:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.777+0000 7fe2b37fe700 1 --2- 172.21.15.78:0/2336213452 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe29c0390f0 0x7fe29c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.777+0000 7fe2b37fe700 1 --2- 172.21.15.78:0/2336213452 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe29c0390f0 0x7fe29c03b5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8006fd0 tx=0x7fe2a8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.782+0000 7fe2b17fa700 1 -- 172.21.15.78:0/2336213452 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe2a4011e20 con 0x7fe2b407e650 2024-04-25T01:36:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.975+0000 7fe2ba48c700 1 -- 172.21.15.78:0/2336213452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fe2b41a1490 con 0x7fe2b407e650 2024-04-25T01:36:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.976+0000 7fe2b17fa700 1 -- 172.21.15.78:0/2336213452 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fe2a4024cd0 con 0x7fe2b407e650 2024-04-25T01:36:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.978+0000 7fe2ba48c700 1 -- 172.21.15.78:0/2336213452 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe29c0390f0 msgr2=0x7fe29c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.978+0000 7fe2ba48c700 1 --2- 172.21.15.78:0/2336213452 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe29c0390f0 0x7fe29c03b5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8006fd0 tx=0x7fe2a8006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.978+0000 7fe2ba48c700 1 -- 172.21.15.78:0/2336213452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2b407e650 msgr2=0x7fe2b419ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.978+0000 7fe2ba48c700 1 --2- 172.21.15.78:0/2336213452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2b407e650 0x7fe2b419ddd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40117e0 tx=0x7fe2a4011c10 comp rx=0 tx=0).stop 2024-04-25T01:36:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.978+0000 7fe2ba48c700 1 -- 172.21.15.78:0/2336213452 shutdown_connections 2024-04-25T01:36:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.978+0000 7fe2ba48c700 1 --2- 172.21.15.78:0/2336213452 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe29c0390f0 0x7fe29c03b5b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.978+0000 7fe2ba48c700 1 --2- 172.21.15.78:0/2336213452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2b407e650 0x7fe2b419ddd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:24.504 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.978+0000 7fe2ba48c700 1 -- 172.21.15.78:0/2336213452 >> 172.21.15.78:0/2336213452 conn(0x7fe2b4103380 msgr2=0x7fe2b4103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:24.505 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.978+0000 7fe2ba48c700 1 -- 172.21.15.78:0/2336213452 shutdown_connections 2024-04-25T01:36:24.505 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:23.978+0000 7fe2ba48c700 1 -- 172.21.15.78:0/2336213452 wait complete. 2024-04-25T01:36:25.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.942+0000 7f40cbd88700 1 Processor -- start 2024-04-25T01:36:25.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.942+0000 7f40cbd88700 1 -- start start 2024-04-25T01:36:25.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.943+0000 7f40cbd88700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c41093c0 0x7f40c41097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:25.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.943+0000 7f40cbd88700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40c41079a0 con 0x7f40c41093c0 2024-04-25T01:36:25.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.943+0000 7f40c9b24700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c41093c0 0x7f40c41097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:25.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.943+0000 7f40c9b24700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c41093c0 0x7f40c41097e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53574/0 (socket says 172.21.15.78:53574) 2024-04-25T01:36:25.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.943+0000 7f40c9b24700 1 -- 172.21.15.78:0/2706482114 learned_addr learned my addr 172.21.15.78:0/2706482114 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:25.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.943+0000 7f40c9b24700 1 -- 172.21.15.78:0/2706482114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c4107b20 con 0x7f40c41093c0 2024-04-25T01:36:25.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.943+0000 7f40c9b24700 1 --2- 172.21.15.78:0/2706482114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c41093c0 0x7f40c41097e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f40c00098a0 tx=0x7f40c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0766c5e1c11d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:25.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.943+0000 7f40c8b22700 1 -- 172.21.15.78:0/2706482114 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c0004160 con 0x7f40c41093c0 2024-04-25T01:36:25.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.943+0000 7f40c8b22700 1 -- 172.21.15.78:0/2706482114 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c0004300 con 0x7f40c41093c0 2024-04-25T01:36:25.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.943+0000 7f40c8b22700 1 -- 172.21.15.78:0/2706482114 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c0011b20 con 0x7f40c41093c0 2024-04-25T01:36:25.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.944+0000 7f40cbd88700 1 -- 172.21.15.78:0/2706482114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c41093c0 msgr2=0x7f40c41097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:25.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.944+0000 7f40cbd88700 1 --2- 172.21.15.78:0/2706482114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c41093c0 0x7f40c41097e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f40c00098a0 tx=0x7f40c0009bb0 comp rx=0 tx=0).stop 2024-04-25T01:36:25.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.944+0000 7f40cbd88700 1 -- 172.21.15.78:0/2706482114 shutdown_connections 2024-04-25T01:36:25.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.944+0000 7f40cbd88700 1 --2- 172.21.15.78:0/2706482114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c41093c0 0x7f40c41097e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:25.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.944+0000 7f40cbd88700 1 -- 172.21.15.78:0/2706482114 >> 172.21.15.78:0/2706482114 conn(0x7f40c4103300 msgr2=0x7f40c4105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:25.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.944+0000 7f40cbd88700 1 -- 172.21.15.78:0/2706482114 shutdown_connections 2024-04-25T01:36:25.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.944+0000 7f40cbd88700 1 -- 172.21.15.78:0/2706482114 wait complete. 2024-04-25T01:36:25.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.945+0000 7f40cbd88700 1 Processor -- start 2024-04-25T01:36:25.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.945+0000 7f40cbd88700 1 -- start start 2024-04-25T01:36:25.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.945+0000 7f40cbd88700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c41093c0 0x7f40c419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:25.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.945+0000 7f40cbd88700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40c410c290 con 0x7f40c41093c0 2024-04-25T01:36:25.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.945+0000 7f40c9b24700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c41093c0 0x7f40c419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:25.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.945+0000 7f40c9b24700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c41093c0 0x7f40c419bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53580/0 (socket says 172.21.15.78:53580) 2024-04-25T01:36:25.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.945+0000 7f40c9b24700 1 -- 172.21.15.78:0/1067833421 learned_addr learned my addr 172.21.15.78:0/1067833421 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:25.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.945+0000 7f40c9b24700 1 -- 172.21.15.78:0/1067833421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c419c260 con 0x7f40c41093c0 2024-04-25T01:36:25.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.945+0000 7f40c9b24700 1 --2- 172.21.15.78:0/1067833421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c41093c0 0x7f40c419bd20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40c00117e0 tx=0x7f40c0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:25.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.945+0000 7f40baffd700 1 -- 172.21.15.78:0/1067833421 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c0003bf0 con 0x7f40c41093c0 2024-04-25T01:36:25.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.945+0000 7f40baffd700 1 -- 172.21.15.78:0/1067833421 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c001a910 con 0x7f40c41093c0 2024-04-25T01:36:25.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.945+0000 7f40cbd88700 1 -- 172.21.15.78:0/1067833421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c419c4f0 con 0x7f40c41093c0 2024-04-25T01:36:25.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.946+0000 7f40baffd700 1 -- 172.21.15.78:0/1067833421 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c00249a0 con 0x7f40c41093c0 2024-04-25T01:36:25.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.946+0000 7f40cbd88700 1 -- 172.21.15.78:0/1067833421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c419c950 con 0x7f40c41093c0 2024-04-25T01:36:25.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.946+0000 7f40baffd700 1 -- 172.21.15.78:0/1067833421 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f40c0022020 con 0x7f40c41093c0 2024-04-25T01:36:25.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.946+0000 7f40cbd88700 1 -- 172.21.15.78:0/1067833421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c4063890 con 0x7f40c41093c0 2024-04-25T01:36:25.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.947+0000 7f40baffd700 1 --2- 172.21.15.78:0/1067833421 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40b0039150 0x7f40b003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:25.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.947+0000 7f40baffd700 1 -- 172.21.15.78:0/1067833421 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f40c004d680 con 0x7f40c41093c0 2024-04-25T01:36:25.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.947+0000 7f40c9323700 1 --2- 172.21.15.78:0/1067833421 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40b0039150 0x7f40b003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:25.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.947+0000 7f40c9323700 1 --2- 172.21.15.78:0/1067833421 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40b0039150 0x7f40b003b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f40b4006fd0 tx=0x7f40b4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:25.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:24.951+0000 7f40baffd700 1 -- 172.21.15.78:0/1067833421 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f40c0011e20 con 0x7f40c41093c0 2024-04-25T01:36:25.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:25.260+0000 7f40cbd88700 1 -- 172.21.15.78:0/1067833421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f40c419f1d0 con 0x7f40c41093c0 2024-04-25T01:36:25.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:25.262+0000 7f40baffd700 1 -- 172.21.15.78:0/1067833421 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f40c419f1d0 con 0x7f40c41093c0 2024-04-25T01:36:25.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:25.264+0000 7f40cbd88700 1 -- 172.21.15.78:0/1067833421 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40b0039150 msgr2=0x7f40b003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:25.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:25.264+0000 7f40cbd88700 1 --2- 172.21.15.78:0/1067833421 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40b0039150 0x7f40b003b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f40b4006fd0 tx=0x7f40b4006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:25.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:25.264+0000 7f40cbd88700 1 -- 172.21.15.78:0/1067833421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c41093c0 msgr2=0x7f40c419bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:25.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:25.264+0000 7f40cbd88700 1 --2- 172.21.15.78:0/1067833421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c41093c0 0x7f40c419bd20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40c00117e0 tx=0x7f40c0011c10 comp rx=0 tx=0).stop 2024-04-25T01:36:25.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:25.264+0000 7f40cbd88700 1 -- 172.21.15.78:0/1067833421 shutdown_connections 2024-04-25T01:36:25.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:25.264+0000 7f40cbd88700 1 --2- 172.21.15.78:0/1067833421 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40b0039150 0x7f40b003b610 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:25.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:25.264+0000 7f40cbd88700 1 --2- 172.21.15.78:0/1067833421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40c41093c0 0x7f40c419bd20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:25.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:25.264+0000 7f40cbd88700 1 -- 172.21.15.78:0/1067833421 >> 172.21.15.78:0/1067833421 conn(0x7f40c4103300 msgr2=0x7f40c4103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:25.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:25.264+0000 7f40cbd88700 1 -- 172.21.15.78:0/1067833421 shutdown_connections 2024-04-25T01:36:25.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-04-25T01:36:25.264+0000 7f40cbd88700 1 -- 172.21.15.78:0/1067833421 wait complete. 2024-04-25T01:36:25.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T01:36:25.745 INFO:teuthology.orchestra.run.smithi078.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T01:36:25.745 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:36:25.745 INFO:teuthology.orchestra.run.smithi078.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:36:25.745 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:36:25.745 INFO:teuthology.orchestra.run.smithi078.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T01:36:25.746 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:36:25.746 INFO:teuthology.orchestra.run.smithi078.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T01:36:25.746 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:36:25.746 INFO:teuthology.orchestra.run.smithi078.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T01:36:25.746 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:36:25.746 INFO:teuthology.orchestra.run.smithi078.stdout: ceph telemetry on 2024-04-25T01:36:25.746 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:36:25.746 INFO:teuthology.orchestra.run.smithi078.stdout:For more information see: 2024-04-25T01:36:25.746 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:36:25.746 INFO:teuthology.orchestra.run.smithi078.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T01:36:25.746 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:36:25.746 INFO:teuthology.orchestra.run.smithi078.stdout:Bootstrap complete. 2024-04-25T01:36:25.777 INFO:tasks.cephadm:Fetching config... 2024-04-25T01:36:25.777 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:36:25.777 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T01:36:25.792 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T01:36:25.792 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:36:25.792 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T01:36:25.846 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T01:36:25.846 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:36:25.847 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T01:36:25.914 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T01:36:25.914 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:36:25.914 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T01:36:25.966 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T01:36:25.966 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFGDsYAnYi1SICvR6km81JBw2ZhEj7TRQYaAnyOMmj08Skwc6Pd22INWVW6AWoPorTbcxYNkCym1Vt1CGKVRupd4T6VO4C1XO+8T/xjepKXR/Q9ams33Dpq8A3cMyLmef66eRsvSpaKoWwO9e3BawuKsfj1Mzdolnh1+kgChZjNxNmAHCwLjRBVeXWHZhl4uxDcuEI/lgBAa+1WVfcUhkkqoUeF6IHJx0HRdDYOImFO8qB+twCGoS01bJ+aQEoMPOqv6+qwS4f4mZGBQ47aBQpc28IV3ujGCQZdLBRRSXV6Ric2bwbcqIZH7Tmida9j8emSPP8mpOL1Y+uvaD25n3nXc6uwl4ZUUtfRi3GMb8vIp0oFLkCBxQwvC/lJ+VN1pLFodDzzJIHZ1jfF/KCAsewQYHRa5iy5DXDxX7WJ9KHcETu654640/krxwxR7WfLukK5kP14sRbkoPQ+6el7mTQSJ6rLyRN+iUMMhNrEUJxx7wf3p0S75seauZPL5g90wE= ceph-df36bea8-02a3-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T01:36:26.039 INFO:teuthology.orchestra.run.smithi078.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFGDsYAnYi1SICvR6km81JBw2ZhEj7TRQYaAnyOMmj08Skwc6Pd22INWVW6AWoPorTbcxYNkCym1Vt1CGKVRupd4T6VO4C1XO+8T/xjepKXR/Q9ams33Dpq8A3cMyLmef66eRsvSpaKoWwO9e3BawuKsfj1Mzdolnh1+kgChZjNxNmAHCwLjRBVeXWHZhl4uxDcuEI/lgBAa+1WVfcUhkkqoUeF6IHJx0HRdDYOImFO8qB+twCGoS01bJ+aQEoMPOqv6+qwS4f4mZGBQ47aBQpc28IV3ujGCQZdLBRRSXV6Ric2bwbcqIZH7Tmida9j8emSPP8mpOL1Y+uvaD25n3nXc6uwl4ZUUtfRi3GMb8vIp0oFLkCBxQwvC/lJ+VN1pLFodDzzJIHZ1jfF/KCAsewQYHRa5iy5DXDxX7WJ9KHcETu654640/krxwxR7WfLukK5kP14sRbkoPQ+6el7mTQSJ6rLyRN+iUMMhNrEUJxx7wf3p0S75seauZPL5g90wE= ceph-df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:36:26.050 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFGDsYAnYi1SICvR6km81JBw2ZhEj7TRQYaAnyOMmj08Skwc6Pd22INWVW6AWoPorTbcxYNkCym1Vt1CGKVRupd4T6VO4C1XO+8T/xjepKXR/Q9ams33Dpq8A3cMyLmef66eRsvSpaKoWwO9e3BawuKsfj1Mzdolnh1+kgChZjNxNmAHCwLjRBVeXWHZhl4uxDcuEI/lgBAa+1WVfcUhkkqoUeF6IHJx0HRdDYOImFO8qB+twCGoS01bJ+aQEoMPOqv6+qwS4f4mZGBQ47aBQpc28IV3ujGCQZdLBRRSXV6Ric2bwbcqIZH7Tmida9j8emSPP8mpOL1Y+uvaD25n3nXc6uwl4ZUUtfRi3GMb8vIp0oFLkCBxQwvC/lJ+VN1pLFodDzzJIHZ1jfF/KCAsewQYHRa5iy5DXDxX7WJ9KHcETu654640/krxwxR7WfLukK5kP14sRbkoPQ+6el7mTQSJ6rLyRN+iUMMhNrEUJxx7wf3p0S75seauZPL5g90wE= ceph-df36bea8-02a3-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T01:36:26.089 INFO:teuthology.orchestra.run.smithi152.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFGDsYAnYi1SICvR6km81JBw2ZhEj7TRQYaAnyOMmj08Skwc6Pd22INWVW6AWoPorTbcxYNkCym1Vt1CGKVRupd4T6VO4C1XO+8T/xjepKXR/Q9ams33Dpq8A3cMyLmef66eRsvSpaKoWwO9e3BawuKsfj1Mzdolnh1+kgChZjNxNmAHCwLjRBVeXWHZhl4uxDcuEI/lgBAa+1WVfcUhkkqoUeF6IHJx0HRdDYOImFO8qB+twCGoS01bJ+aQEoMPOqv6+qwS4f4mZGBQ47aBQpc28IV3ujGCQZdLBRRSXV6Ric2bwbcqIZH7Tmida9j8emSPP8mpOL1Y+uvaD25n3nXc6uwl4ZUUtfRi3GMb8vIp0oFLkCBxQwvC/lJ+VN1pLFodDzzJIHZ1jfF/KCAsewQYHRa5iy5DXDxX7WJ9KHcETu654640/krxwxR7WfLukK5kP14sRbkoPQ+6el7mTQSJ6rLyRN+iUMMhNrEUJxx7wf3p0S75seauZPL5g90wE= ceph-df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:36:26.100 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFGDsYAnYi1SICvR6km81JBw2ZhEj7TRQYaAnyOMmj08Skwc6Pd22INWVW6AWoPorTbcxYNkCym1Vt1CGKVRupd4T6VO4C1XO+8T/xjepKXR/Q9ams33Dpq8A3cMyLmef66eRsvSpaKoWwO9e3BawuKsfj1Mzdolnh1+kgChZjNxNmAHCwLjRBVeXWHZhl4uxDcuEI/lgBAa+1WVfcUhkkqoUeF6IHJx0HRdDYOImFO8qB+twCGoS01bJ+aQEoMPOqv6+qwS4f4mZGBQ47aBQpc28IV3ujGCQZdLBRRSXV6Ric2bwbcqIZH7Tmida9j8emSPP8mpOL1Y+uvaD25n3nXc6uwl4ZUUtfRi3GMb8vIp0oFLkCBxQwvC/lJ+VN1pLFodDzzJIHZ1jfF/KCAsewQYHRa5iy5DXDxX7WJ9KHcETu654640/krxwxR7WfLukK5kP14sRbkoPQ+6el7mTQSJ6rLyRN+iUMMhNrEUJxx7wf3p0S75seauZPL5g90wE= ceph-df36bea8-02a3-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T01:36:26.139 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFGDsYAnYi1SICvR6km81JBw2ZhEj7TRQYaAnyOMmj08Skwc6Pd22INWVW6AWoPorTbcxYNkCym1Vt1CGKVRupd4T6VO4C1XO+8T/xjepKXR/Q9ams33Dpq8A3cMyLmef66eRsvSpaKoWwO9e3BawuKsfj1Mzdolnh1+kgChZjNxNmAHCwLjRBVeXWHZhl4uxDcuEI/lgBAa+1WVfcUhkkqoUeF6IHJx0HRdDYOImFO8qB+twCGoS01bJ+aQEoMPOqv6+qwS4f4mZGBQ47aBQpc28IV3ujGCQZdLBRRSXV6Ric2bwbcqIZH7Tmida9j8emSPP8mpOL1Y+uvaD25n3nXc6uwl4ZUUtfRi3GMb8vIp0oFLkCBxQwvC/lJ+VN1pLFodDzzJIHZ1jfF/KCAsewQYHRa5iy5DXDxX7WJ9KHcETu654640/krxwxR7WfLukK5kP14sRbkoPQ+6el7mTQSJ6rLyRN+iUMMhNrEUJxx7wf3p0S75seauZPL5g90wE= ceph-df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:36:26.151 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T01:36:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:36:26.491 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:26 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1067833421' entity='client.admin' 2024-04-25T01:36:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.562+0000 7f2c5db42700 1 -- 172.21.15.78:0/723008700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c58102720 msgr2=0x7f2c58104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.562+0000 7f2c5db42700 1 --2- 172.21.15.78:0/723008700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c58102720 0x7f2c58104b10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009910 tx=0x7f2c48009c20 comp rx=0 tx=0).stop 2024-04-25T01:36:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.562+0000 7f2c5db42700 1 -- 172.21.15.78:0/723008700 shutdown_connections 2024-04-25T01:36:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.562+0000 7f2c5db42700 1 --2- 172.21.15.78:0/723008700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c58102720 0x7f2c58104b10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.562+0000 7f2c5db42700 1 -- 172.21.15.78:0/723008700 >> 172.21.15.78:0/723008700 conn(0x7f2c580fc130 msgr2=0x7f2c580fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.562+0000 7f2c5db42700 1 -- 172.21.15.78:0/723008700 shutdown_connections 2024-04-25T01:36:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.562+0000 7f2c5db42700 1 -- 172.21.15.78:0/723008700 wait complete. 2024-04-25T01:36:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.562+0000 7f2c5db42700 1 Processor -- start 2024-04-25T01:36:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.562+0000 7f2c5db42700 1 -- start start 2024-04-25T01:36:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.563+0000 7f2c5db42700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c58102720 0x7f2c58197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:28.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.563+0000 7f2c5db42700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c58107cc0 con 0x7f2c58102720 2024-04-25T01:36:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.563+0000 7f2c577fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c58102720 0x7f2c58197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.563+0000 7f2c577fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c58102720 0x7f2c58197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53604/0 (socket says 172.21.15.78:53604) 2024-04-25T01:36:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.563+0000 7f2c577fe700 1 -- 172.21.15.78:0/3561449932 learned_addr learned my addr 172.21.15.78:0/3561449932 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.563+0000 7f2c577fe700 1 -- 172.21.15.78:0/3561449932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c58197840 con 0x7f2c58102720 2024-04-25T01:36:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.563+0000 7f2c577fe700 1 --2- 172.21.15.78:0/3561449932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c58102720 0x7f2c58197300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2c48004fa0 tx=0x7f2c48005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.563+0000 7f2c54ff9700 1 -- 172.21.15.78:0/3561449932 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c48027070 con 0x7f2c58102720 2024-04-25T01:36:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.563+0000 7f2c54ff9700 1 -- 172.21.15.78:0/3561449932 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c4800be40 con 0x7f2c58102720 2024-04-25T01:36:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.563+0000 7f2c5db42700 1 -- 172.21.15.78:0/3561449932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c58197b30 con 0x7f2c58102720 2024-04-25T01:36:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.563+0000 7f2c54ff9700 1 -- 172.21.15.78:0/3561449932 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c4800f460 con 0x7f2c58102720 2024-04-25T01:36:28.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.563+0000 7f2c5db42700 1 -- 172.21.15.78:0/3561449932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c5819a7c0 con 0x7f2c58102720 2024-04-25T01:36:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.564+0000 7f2c54ff9700 1 -- 172.21.15.78:0/3561449932 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2c4802a070 con 0x7f2c58102720 2024-04-25T01:36:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.564+0000 7f2c5db42700 1 -- 172.21.15.78:0/3561449932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c58108440 con 0x7f2c58102720 2024-04-25T01:36:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.565+0000 7f2c54ff9700 1 --2- 172.21.15.78:0/3561449932 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2c40039150 0x7f2c4003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:28.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.565+0000 7f2c54ff9700 1 -- 172.21.15.78:0/3561449932 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2c4801e030 con 0x7f2c58102720 2024-04-25T01:36:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.565+0000 7f2c56ffd700 1 --2- 172.21.15.78:0/3561449932 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2c40039150 0x7f2c4003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:28.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.565+0000 7f2c56ffd700 1 --2- 172.21.15.78:0/3561449932 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2c40039150 0x7f2c4003b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c006fd0 tx=0x7f2c4c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.570+0000 7f2c54ff9700 1 -- 172.21.15.78:0/3561449932 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2c48013070 con 0x7f2c58102720 2024-04-25T01:36:28.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.773+0000 7f2c5db42700 1 -- 172.21.15.78:0/3561449932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f2c5802dcb0 con 0x7f2c58102720 2024-04-25T01:36:28.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.775+0000 7f2c54ff9700 1 -- 172.21.15.78:0/3561449932 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f2c480224e0 con 0x7f2c58102720 2024-04-25T01:36:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.777+0000 7f2c5db42700 1 -- 172.21.15.78:0/3561449932 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2c40039150 msgr2=0x7f2c4003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.777+0000 7f2c5db42700 1 --2- 172.21.15.78:0/3561449932 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2c40039150 0x7f2c4003b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c006fd0 tx=0x7f2c4c006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.777+0000 7f2c5db42700 1 -- 172.21.15.78:0/3561449932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c58102720 msgr2=0x7f2c58197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.777+0000 7f2c5db42700 1 --2- 172.21.15.78:0/3561449932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c58102720 0x7f2c58197300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2c48004fa0 tx=0x7f2c48005080 comp rx=0 tx=0).stop 2024-04-25T01:36:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.777+0000 7f2c5db42700 1 -- 172.21.15.78:0/3561449932 shutdown_connections 2024-04-25T01:36:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.777+0000 7f2c5db42700 1 --2- 172.21.15.78:0/3561449932 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2c40039150 0x7f2c4003b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.777+0000 7f2c5db42700 1 --2- 172.21.15.78:0/3561449932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c58102720 0x7f2c58197300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.777+0000 7f2c5db42700 1 -- 172.21.15.78:0/3561449932 >> 172.21.15.78:0/3561449932 conn(0x7f2c580fc130 msgr2=0x7f2c580fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:28.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.778+0000 7f2c5db42700 1 -- 172.21.15.78:0/3561449932 shutdown_connections 2024-04-25T01:36:28.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:28.778+0000 7f2c5db42700 1 -- 172.21.15.78:0/3561449932 wait complete. 2024-04-25T01:36:28.841 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:28 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:28.841 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:28 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:28.841 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:28 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:28.841 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:28 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:28.842 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:28 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:36:28.842 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:28 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:29.405 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T01:36:29.405 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T01:36:29.634 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:36:30.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:29 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3561449932' entity='client.admin' 2024-04-25T01:36:30.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:36:30.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:31.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.075+0000 7f17236ff700 1 -- 172.21.15.78:0/873148065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f171c100b90 msgr2=0x7f171c100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:31.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.075+0000 7f17236ff700 1 --2- 172.21.15.78:0/873148065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f171c100b90 0x7f171c100fb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1718009910 tx=0x7f1718009c20 comp rx=0 tx=0).stop 2024-04-25T01:36:31.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.075+0000 7f17236ff700 1 -- 172.21.15.78:0/873148065 shutdown_connections 2024-04-25T01:36:31.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.075+0000 7f17236ff700 1 --2- 172.21.15.78:0/873148065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f171c100b90 0x7f171c100fb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:31.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.075+0000 7f17236ff700 1 -- 172.21.15.78:0/873148065 >> 172.21.15.78:0/873148065 conn(0x7f171c0fc130 msgr2=0x7f171c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:31.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.075+0000 7f17236ff700 1 -- 172.21.15.78:0/873148065 shutdown_connections 2024-04-25T01:36:31.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.075+0000 7f17236ff700 1 -- 172.21.15.78:0/873148065 wait complete. 2024-04-25T01:36:31.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.075+0000 7f17236ff700 1 Processor -- start 2024-04-25T01:36:31.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.076+0000 7f17236ff700 1 -- start start 2024-04-25T01:36:31.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.076+0000 7f17236ff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f171c100b90 0x7f171c06a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:31.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.076+0000 7f17236ff700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f171c106c20 con 0x7f171c100b90 2024-04-25T01:36:31.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.076+0000 7f172149b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f171c100b90 0x7f171c06a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:31.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.076+0000 7f172149b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f171c100b90 0x7f171c06a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46832/0 (socket says 172.21.15.78:46832) 2024-04-25T01:36:31.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.076+0000 7f172149b700 1 -- 172.21.15.78:0/1431590493 learned_addr learned my addr 172.21.15.78:0/1431590493 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:31.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.076+0000 7f172149b700 1 -- 172.21.15.78:0/1431590493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c19a0d0 con 0x7f171c100b90 2024-04-25T01:36:31.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.076+0000 7f172149b700 1 --2- 172.21.15.78:0/1431590493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f171c100b90 0x7f171c06a0f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1718006010 tx=0x7f1718005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:31.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.076+0000 7f17127fc700 1 -- 172.21.15.78:0/1431590493 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1718027070 con 0x7f171c100b90 2024-04-25T01:36:31.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.076+0000 7f17236ff700 1 -- 172.21.15.78:0/1431590493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171c19a2a0 con 0x7f171c100b90 2024-04-25T01:36:31.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.076+0000 7f17127fc700 1 -- 172.21.15.78:0/1431590493 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f171800be40 con 0x7f171c100b90 2024-04-25T01:36:31.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.076+0000 7f17127fc700 1 -- 172.21.15.78:0/1431590493 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f171800f460 con 0x7f171c100b90 2024-04-25T01:36:31.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.076+0000 7f17236ff700 1 -- 172.21.15.78:0/1431590493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f171c19a6c0 con 0x7f171c100b90 2024-04-25T01:36:31.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.077+0000 7f17127fc700 1 -- 172.21.15.78:0/1431590493 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f171802a070 con 0x7f171c100b90 2024-04-25T01:36:31.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.077+0000 7f17236ff700 1 -- 172.21.15.78:0/1431590493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171c1073a0 con 0x7f171c100b90 2024-04-25T01:36:31.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.078+0000 7f17127fc700 1 --2- 172.21.15.78:0/1431590493 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1708039150 0x7f170803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:31.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.078+0000 7f17127fc700 1 -- 172.21.15.78:0/1431590493 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f171801e030 con 0x7f171c100b90 2024-04-25T01:36:31.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.078+0000 7f1720c9a700 1 --2- 172.21.15.78:0/1431590493 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1708039150 0x7f170803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:31.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.078+0000 7f1720c9a700 1 --2- 172.21.15.78:0/1431590493 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1708039150 0x7f170803b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f170c006fd0 tx=0x7f170c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:31.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.082+0000 7f17127fc700 1 -- 172.21.15.78:0/1431590493 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1718013070 con 0x7f171c100b90 2024-04-25T01:36:31.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.284+0000 7f17236ff700 1 -- 172.21.15.78:0/1431590493 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f171c02ddf0 con 0x7f1708039150 2024-04-25T01:36:31.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.287+0000 7f17127fc700 1 -- 172.21.15.78:0/1431590493 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f171c02ddf0 con 0x7f1708039150 2024-04-25T01:36:31.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.289+0000 7f17236ff700 1 -- 172.21.15.78:0/1431590493 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1708039150 msgr2=0x7f170803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:31.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.289+0000 7f17236ff700 1 --2- 172.21.15.78:0/1431590493 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1708039150 0x7f170803b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f170c006fd0 tx=0x7f170c006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:31.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.289+0000 7f17236ff700 1 -- 172.21.15.78:0/1431590493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f171c100b90 msgr2=0x7f171c06a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:31.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.289+0000 7f17236ff700 1 --2- 172.21.15.78:0/1431590493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f171c100b90 0x7f171c06a0f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1718006010 tx=0x7f1718005050 comp rx=0 tx=0).stop 2024-04-25T01:36:31.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.289+0000 7f17236ff700 1 -- 172.21.15.78:0/1431590493 shutdown_connections 2024-04-25T01:36:31.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.289+0000 7f17236ff700 1 --2- 172.21.15.78:0/1431590493 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1708039150 0x7f170803b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:31.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.289+0000 7f17236ff700 1 --2- 172.21.15.78:0/1431590493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f171c100b90 0x7f171c06a0f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:31.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.289+0000 7f17236ff700 1 -- 172.21.15.78:0/1431590493 >> 172.21.15.78:0/1431590493 conn(0x7f171c0fc130 msgr2=0x7f171c0fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:31.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.289+0000 7f17236ff700 1 -- 172.21.15.78:0/1431590493 shutdown_connections 2024-04-25T01:36:31.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:31.289+0000 7f17236ff700 1 -- 172.21.15.78:0/1431590493 wait complete. 2024-04-25T01:36:32.000 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi152 2024-04-25T01:36:32.000 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T01:36:32.000 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/etc/ceph/ceph.conf 2024-04-25T01:36:32.019 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T01:36:32.019 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:36:32.077 INFO:tasks.cephadm:Adding host smithi152 to orchestrator... 2024-04-25T01:36:32.077 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch host add smithi152 2024-04-25T01:36:32.322 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:36:32.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:32 smithi078 ceph-mon[29469]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:32.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:32 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:32.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:32 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:36:32.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:32 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:36:32.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:32 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:36:32.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:32 smithi078 ceph-mon[29469]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-25T01:36:32.549 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:32 smithi078 ceph-mon[29469]: Updating smithi078:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:36:32.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:32 smithi078 ceph-mon[29469]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:36:32.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:32 smithi078 ceph-mon[29469]: Updating smithi078:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:36:32.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:32 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:32.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:32 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:32.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:32 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:33.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.914+0000 7fc7f0f64700 1 -- 172.21.15.78:0/4097577004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7ec101ea0 msgr2=0x7fc7ec102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:33.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.914+0000 7fc7f0f64700 1 --2- 172.21.15.78:0/4097577004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7ec101ea0 0x7fc7ec102280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009910 tx=0x7fc7dc009c20 comp rx=0 tx=0).stop 2024-04-25T01:36:33.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.915+0000 7fc7f0f64700 1 -- 172.21.15.78:0/4097577004 shutdown_connections 2024-04-25T01:36:33.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.915+0000 7fc7f0f64700 1 --2- 172.21.15.78:0/4097577004 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7ec101ea0 0x7fc7ec102280 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:33.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.915+0000 7fc7f0f64700 1 -- 172.21.15.78:0/4097577004 >> 172.21.15.78:0/4097577004 conn(0x7fc7ec0fd770 msgr2=0x7fc7ec0ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:33.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.915+0000 7fc7f0f64700 1 -- 172.21.15.78:0/4097577004 shutdown_connections 2024-04-25T01:36:33.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.915+0000 7fc7f0f64700 1 -- 172.21.15.78:0/4097577004 wait complete. 2024-04-25T01:36:33.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.915+0000 7fc7f0f64700 1 Processor -- start 2024-04-25T01:36:33.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.915+0000 7fc7f0f64700 1 -- start start 2024-04-25T01:36:33.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.916+0000 7fc7f0f64700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7ec101ea0 0x7fc7ec074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:33.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.916+0000 7fc7f0f64700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec10c270 con 0x7fc7ec101ea0 2024-04-25T01:36:33.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.916+0000 7fc7ea59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7ec101ea0 0x7fc7ec074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:33.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.916+0000 7fc7ea59c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7ec101ea0 0x7fc7ec074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46850/0 (socket says 172.21.15.78:46850) 2024-04-25T01:36:33.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.916+0000 7fc7ea59c700 1 -- 172.21.15.78:0/556445424 learned_addr learned my addr 172.21.15.78:0/556445424 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:33.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.916+0000 7fc7ea59c700 1 -- 172.21.15.78:0/556445424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec076420 con 0x7fc7ec101ea0 2024-04-25T01:36:33.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.916+0000 7fc7ea59c700 1 --2- 172.21.15.78:0/556445424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7ec101ea0 0x7fc7ec074530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc006010 tx=0x7fc7dc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:33.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.916+0000 7fc7db7fe700 1 -- 172.21.15.78:0/556445424 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7dc026070 con 0x7fc7ec101ea0 2024-04-25T01:36:33.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.916+0000 7fc7db7fe700 1 -- 172.21.15.78:0/556445424 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7dc00bc30 con 0x7fc7ec101ea0 2024-04-25T01:36:33.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.916+0000 7fc7f0f64700 1 -- 172.21.15.78:0/556445424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ec074ad0 con 0x7fc7ec101ea0 2024-04-25T01:36:33.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.916+0000 7fc7db7fe700 1 -- 172.21.15.78:0/556445424 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7dc0039d0 con 0x7fc7ec101ea0 2024-04-25T01:36:33.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.916+0000 7fc7f0f64700 1 -- 172.21.15.78:0/556445424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ec074f90 con 0x7fc7ec101ea0 2024-04-25T01:36:33.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.917+0000 7fc7db7fe700 1 -- 172.21.15.78:0/556445424 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc7dc029070 con 0x7fc7ec101ea0 2024-04-25T01:36:33.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.917+0000 7fc7db7fe700 1 --2- 172.21.15.78:0/556445424 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc7d40390f0 0x7fc7d403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:33.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.917+0000 7fc7db7fe700 1 -- 172.21.15.78:0/556445424 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc7dc01e070 con 0x7fc7ec101ea0 2024-04-25T01:36:33.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.917+0000 7fc7f0f64700 1 -- 172.21.15.78:0/556445424 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ec103380 con 0x7fc7ec101ea0 2024-04-25T01:36:33.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.917+0000 7fc7e9d9b700 1 --2- 172.21.15.78:0/556445424 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc7d40390f0 0x7fc7d403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:33.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.918+0000 7fc7e9d9b700 1 --2- 172.21.15.78:0/556445424 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc7d40390f0 0x7fc7d403b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0006fd0 tx=0x7fc7e0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:33.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:33.922+0000 7fc7db7fe700 1 -- 172.21.15.78:0/556445424 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc7dc07b740 con 0x7fc7ec101ea0 2024-04-25T01:36:34.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:34.121+0000 7fc7f0f64700 1 -- 172.21.15.78:0/556445424 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi152", "target": ["mon-mgr", ""]}) -- 0x7fc7ec075a60 con 0x7fc7d40390f0 2024-04-25T01:36:35.491 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:35 smithi078 ceph-mon[29469]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi152", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:35.491 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:35 smithi078 ceph-mon[29469]: Deploying cephadm binary to smithi152 2024-04-25T01:36:38.133 INFO:teuthology.orchestra.run.smithi078.stdout:Added host 'smithi152' with addr '172.21.15.152' 2024-04-25T01:36:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:38.157+0000 7fc7db7fe700 1 -- 172.21.15.78:0/556445424 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc7ec075a60 con 0x7fc7d40390f0 2024-04-25T01:36:38.134 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:37 smithi078 ceph-mon[29469]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:36:38.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:38.159+0000 7fc7f0f64700 1 -- 172.21.15.78:0/556445424 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc7d40390f0 msgr2=0x7fc7d403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:38.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:38.159+0000 7fc7f0f64700 1 --2- 172.21.15.78:0/556445424 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc7d40390f0 0x7fc7d403b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0006fd0 tx=0x7fc7e0006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:38.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:38.159+0000 7fc7f0f64700 1 -- 172.21.15.78:0/556445424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7ec101ea0 msgr2=0x7fc7ec074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:38.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:38.159+0000 7fc7f0f64700 1 --2- 172.21.15.78:0/556445424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7ec101ea0 0x7fc7ec074530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc006010 tx=0x7fc7dc005030 comp rx=0 tx=0).stop 2024-04-25T01:36:38.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:38.159+0000 7fc7f0f64700 1 -- 172.21.15.78:0/556445424 shutdown_connections 2024-04-25T01:36:38.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:38.159+0000 7fc7f0f64700 1 --2- 172.21.15.78:0/556445424 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc7d40390f0 0x7fc7d403b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:38.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:38.159+0000 7fc7f0f64700 1 --2- 172.21.15.78:0/556445424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7ec101ea0 0x7fc7ec074530 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:38.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:38.159+0000 7fc7f0f64700 1 -- 172.21.15.78:0/556445424 >> 172.21.15.78:0/556445424 conn(0x7fc7ec0fd770 msgr2=0x7fc7ec104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:38.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:38.159+0000 7fc7f0f64700 1 -- 172.21.15.78:0/556445424 shutdown_connections 2024-04-25T01:36:38.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:38.160+0000 7fc7f0f64700 1 -- 172.21.15.78:0/556445424 wait complete. 2024-04-25T01:36:38.712 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T01:36:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:36:39.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:39.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:39 smithi078 ceph-mon[29469]: Added host smithi152 2024-04-25T01:36:39.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:36:39.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:39.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:39 smithi078 ceph-mon[29469]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:36:40.434 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:40 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:40.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.558+0000 7f02012ad700 1 -- 172.21.15.78:0/1222128777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01fc101ea0 msgr2=0x7f01fc102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:40.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.558+0000 7f02012ad700 1 --2- 172.21.15.78:0/1222128777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01fc101ea0 0x7f01fc102280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009910 tx=0x7f01f0009c20 comp rx=0 tx=0).stop 2024-04-25T01:36:40.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.558+0000 7f02012ad700 1 -- 172.21.15.78:0/1222128777 shutdown_connections 2024-04-25T01:36:40.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.558+0000 7f02012ad700 1 --2- 172.21.15.78:0/1222128777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01fc101ea0 0x7f01fc102280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:40.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.558+0000 7f02012ad700 1 -- 172.21.15.78:0/1222128777 >> 172.21.15.78:0/1222128777 conn(0x7f01fc0fd770 msgr2=0x7f01fc0ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:40.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.558+0000 7f02012ad700 1 -- 172.21.15.78:0/1222128777 shutdown_connections 2024-04-25T01:36:40.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.558+0000 7f02012ad700 1 -- 172.21.15.78:0/1222128777 wait complete. 2024-04-25T01:36:40.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.558+0000 7f02012ad700 1 Processor -- start 2024-04-25T01:36:40.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.559+0000 7f02012ad700 1 -- start start 2024-04-25T01:36:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.559+0000 7f02012ad700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01fc101ea0 0x7f01fc074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.559+0000 7f02012ad700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc10c270 con 0x7f01fc101ea0 2024-04-25T01:36:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.559+0000 7f01faffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01fc101ea0 0x7f01fc074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.559+0000 7f01faffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01fc101ea0 0x7f01fc074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41838/0 (socket says 172.21.15.78:41838) 2024-04-25T01:36:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.559+0000 7f01faffd700 1 -- 172.21.15.78:0/3479711275 learned_addr learned my addr 172.21.15.78:0/3479711275 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.559+0000 7f01faffd700 1 -- 172.21.15.78:0/3479711275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01fc076420 con 0x7f01fc101ea0 2024-04-25T01:36:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.559+0000 7f01faffd700 1 --2- 172.21.15.78:0/3479711275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01fc101ea0 0x7f01fc074530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f01f0005850 tx=0x7f01f0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.560+0000 7f01e3fff700 1 -- 172.21.15.78:0/3479711275 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01f0026070 con 0x7f01fc101ea0 2024-04-25T01:36:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.560+0000 7f01e3fff700 1 -- 172.21.15.78:0/3479711275 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f000bc30 con 0x7f01fc101ea0 2024-04-25T01:36:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.560+0000 7f01e3fff700 1 -- 172.21.15.78:0/3479711275 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01f0003a20 con 0x7f01fc101ea0 2024-04-25T01:36:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.560+0000 7f02012ad700 1 -- 172.21.15.78:0/3479711275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01fc074ad0 con 0x7f01fc101ea0 2024-04-25T01:36:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.560+0000 7f02012ad700 1 -- 172.21.15.78:0/3479711275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01fc074f90 con 0x7f01fc101ea0 2024-04-25T01:36:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.560+0000 7f01e3fff700 1 -- 172.21.15.78:0/3479711275 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f01f0029070 con 0x7f01fc101ea0 2024-04-25T01:36:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.561+0000 7f02012ad700 1 -- 172.21.15.78:0/3479711275 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01fc103380 con 0x7f01fc101ea0 2024-04-25T01:36:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.561+0000 7f01e3fff700 1 --2- 172.21.15.78:0/3479711275 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f01e4039150 0x7f01e403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.561+0000 7f01e3fff700 1 -- 172.21.15.78:0/3479711275 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f01f001e070 con 0x7f01fc101ea0 2024-04-25T01:36:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.561+0000 7f01fa7fc700 1 --2- 172.21.15.78:0/3479711275 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f01e4039150 0x7f01e403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.561+0000 7f01fa7fc700 1 --2- 172.21.15.78:0/3479711275 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f01e4039150 0x7f01e403b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f01ec006fd0 tx=0x7f01ec006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.566+0000 7f01e3fff700 1 -- 172.21.15.78:0/3479711275 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f01f0013070 con 0x7f01fc101ea0 2024-04-25T01:36:40.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.768+0000 7f02012ad700 1 -- 172.21.15.78:0/3479711275 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01fc075ad0 con 0x7f01e4039150 2024-04-25T01:36:40.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.769+0000 7f01e3fff700 1 -- 172.21.15.78:0/3479711275 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f01fc075ad0 con 0x7f01e4039150 2024-04-25T01:36:40.745 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:36:40.745 INFO:teuthology.orchestra.run.smithi078.stdout:[{"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.152", "hostname": "smithi152", "labels": [], "status": ""}] 2024-04-25T01:36:40.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.771+0000 7f02012ad700 1 -- 172.21.15.78:0/3479711275 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f01e4039150 msgr2=0x7f01e403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:40.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.771+0000 7f02012ad700 1 --2- 172.21.15.78:0/3479711275 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f01e4039150 0x7f01e403b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f01ec006fd0 tx=0x7f01ec006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:40.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.771+0000 7f02012ad700 1 -- 172.21.15.78:0/3479711275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01fc101ea0 msgr2=0x7f01fc074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:40.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.771+0000 7f02012ad700 1 --2- 172.21.15.78:0/3479711275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01fc101ea0 0x7f01fc074530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f01f0005850 tx=0x7f01f0005030 comp rx=0 tx=0).stop 2024-04-25T01:36:40.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.771+0000 7f02012ad700 1 -- 172.21.15.78:0/3479711275 shutdown_connections 2024-04-25T01:36:40.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.771+0000 7f02012ad700 1 --2- 172.21.15.78:0/3479711275 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f01e4039150 0x7f01e403b610 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:40.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.771+0000 7f02012ad700 1 --2- 172.21.15.78:0/3479711275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01fc101ea0 0x7f01fc074530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:40.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.771+0000 7f02012ad700 1 -- 172.21.15.78:0/3479711275 >> 172.21.15.78:0/3479711275 conn(0x7f01fc0fd770 msgr2=0x7f01fc103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:40.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.771+0000 7f02012ad700 1 -- 172.21.15.78:0/3479711275 shutdown_connections 2024-04-25T01:36:40.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:40.771+0000 7f02012ad700 1 -- 172.21.15.78:0/3479711275 wait complete. 2024-04-25T01:36:41.347 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-04-25T01:36:41.347 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T01:36:41.347 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-04-25T01:36:41.365 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T01:36:41.365 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:36:41.422 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-04-25T01:36:41.423 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch host add smithi196 2024-04-25T01:36:41.653 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:36:42.073 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:41 smithi078 ceph-mon[29469]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:36:42.073 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:42.073 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:41 smithi078 ceph-mon[29469]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:36:43.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.191+0000 7f673fa06700 1 -- 172.21.15.78:0/3377855024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6738106c00 msgr2=0x7f6738106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:43.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.191+0000 7f673fa06700 1 --2- 172.21.15.78:0/3377855024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6738106c00 0x7f6738106fe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6728009910 tx=0x7f6728009c20 comp rx=0 tx=0).stop 2024-04-25T01:36:43.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.191+0000 7f673fa06700 1 -- 172.21.15.78:0/3377855024 shutdown_connections 2024-04-25T01:36:43.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.191+0000 7f673fa06700 1 --2- 172.21.15.78:0/3377855024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6738106c00 0x7f6738106fe0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:43.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.191+0000 7f673fa06700 1 -- 172.21.15.78:0/3377855024 >> 172.21.15.78:0/3377855024 conn(0x7f6738076690 msgr2=0x7f6738076aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:43.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.191+0000 7f673fa06700 1 -- 172.21.15.78:0/3377855024 shutdown_connections 2024-04-25T01:36:43.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.191+0000 7f673fa06700 1 -- 172.21.15.78:0/3377855024 wait complete. 2024-04-25T01:36:43.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.192+0000 7f673fa06700 1 Processor -- start 2024-04-25T01:36:43.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.192+0000 7f673fa06700 1 -- start start 2024-04-25T01:36:43.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.192+0000 7f673fa06700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6738106c00 0x7f6738198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:43.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.192+0000 7f673fa06700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f673810b1c0 con 0x7f6738106c00 2024-04-25T01:36:43.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.192+0000 7f673d7a2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6738106c00 0x7f6738198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:43.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.192+0000 7f673d7a2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6738106c00 0x7f6738198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41858/0 (socket says 172.21.15.78:41858) 2024-04-25T01:36:43.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.192+0000 7f673d7a2700 1 -- 172.21.15.78:0/866621033 learned_addr learned my addr 172.21.15.78:0/866621033 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:43.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.192+0000 7f673d7a2700 1 -- 172.21.15.78:0/866621033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6738198ab0 con 0x7f6738106c00 2024-04-25T01:36:43.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.192+0000 7f673d7a2700 1 --2- 172.21.15.78:0/866621033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6738106c00 0x7f6738198570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6728005850 tx=0x7f6728005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:43.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.192+0000 7f672e7fc700 1 -- 172.21.15.78:0/866621033 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6728026070 con 0x7f6738106c00 2024-04-25T01:36:43.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.192+0000 7f672e7fc700 1 -- 172.21.15.78:0/866621033 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672800bc30 con 0x7f6738106c00 2024-04-25T01:36:43.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.193+0000 7f673fa06700 1 -- 172.21.15.78:0/866621033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6738198d40 con 0x7f6738106c00 2024-04-25T01:36:43.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.193+0000 7f672e7fc700 1 -- 172.21.15.78:0/866621033 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6728003a20 con 0x7f6738106c00 2024-04-25T01:36:43.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.193+0000 7f673fa06700 1 -- 172.21.15.78:0/866621033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67381991a0 con 0x7f6738106c00 2024-04-25T01:36:43.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.193+0000 7f672e7fc700 1 -- 172.21.15.78:0/866621033 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6728029070 con 0x7f6738106c00 2024-04-25T01:36:43.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.193+0000 7f673fa06700 1 -- 172.21.15.78:0/866621033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6738102200 con 0x7f6738106c00 2024-04-25T01:36:43.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.194+0000 7f672e7fc700 1 --2- 172.21.15.78:0/866621033 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6724039150 0x7f672403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:43.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.194+0000 7f672e7fc700 1 -- 172.21.15.78:0/866621033 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f672801e070 con 0x7f6738106c00 2024-04-25T01:36:43.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.194+0000 7f673cfa1700 1 --2- 172.21.15.78:0/866621033 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6724039150 0x7f672403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:43.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.194+0000 7f673cfa1700 1 --2- 172.21.15.78:0/866621033 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6724039150 0x7f672403b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6734006fd0 tx=0x7f6734006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:43.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.198+0000 7f672e7fc700 1 -- 172.21.15.78:0/866621033 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6728013070 con 0x7f6738106c00 2024-04-25T01:36:43.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:43.398+0000 7f673fa06700 1 -- 172.21.15.78:0/866621033 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}) -- 0x7f6738069c40 con 0x7f6724039150 2024-04-25T01:36:44.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:43 smithi078 ceph-mon[29469]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:36:45.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:44 smithi078 ceph-mon[29469]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:45.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:44 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:45.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:44 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:45.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:44 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:45.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:44 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:45.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:44 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi152", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:45.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:44 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:36:45.183 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:44 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:36:45.183 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:44 smithi078 ceph-mon[29469]: Updating smithi152:/etc/ceph/ceph.conf 2024-04-25T01:36:46.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:45 smithi078 ceph-mon[29469]: Deploying cephadm binary to smithi196 2024-04-25T01:36:46.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:45 smithi078 ceph-mon[29469]: Updating smithi152:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:36:46.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:45 smithi078 ceph-mon[29469]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:36:46.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:45 smithi078 ceph-mon[29469]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:36:47.518 INFO:teuthology.orchestra.run.smithi078.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-04-25T01:36:47.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:47.541+0000 7f672e7fc700 1 -- 172.21.15.78:0/866621033 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6738069c40 con 0x7f6724039150 2024-04-25T01:36:47.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:47.544+0000 7f673fa06700 1 -- 172.21.15.78:0/866621033 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6724039150 msgr2=0x7f672403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:47.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:47.544+0000 7f673fa06700 1 --2- 172.21.15.78:0/866621033 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6724039150 0x7f672403b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6734006fd0 tx=0x7f6734006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:47.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:47.544+0000 7f673fa06700 1 -- 172.21.15.78:0/866621033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6738106c00 msgr2=0x7f6738198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:47.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:47.544+0000 7f673fa06700 1 --2- 172.21.15.78:0/866621033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6738106c00 0x7f6738198570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6728005850 tx=0x7f6728005030 comp rx=0 tx=0).stop 2024-04-25T01:36:47.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:47.544+0000 7f673fa06700 1 -- 172.21.15.78:0/866621033 shutdown_connections 2024-04-25T01:36:47.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:47.544+0000 7f673fa06700 1 --2- 172.21.15.78:0/866621033 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6724039150 0x7f672403b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:47.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:47.544+0000 7f673fa06700 1 --2- 172.21.15.78:0/866621033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6738106c00 0x7f6738198570 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:47.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:47.544+0000 7f673fa06700 1 -- 172.21.15.78:0/866621033 >> 172.21.15.78:0/866621033 conn(0x7f6738076690 msgr2=0x7f67380ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:47.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:47.544+0000 7f673fa06700 1 -- 172.21.15.78:0/866621033 shutdown_connections 2024-04-25T01:36:47.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:47.544+0000 7f673fa06700 1 -- 172.21.15.78:0/866621033 wait complete. 2024-04-25T01:36:48.107 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:47 smithi078 ceph-mon[29469]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:36:48.107 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:47 smithi078 ceph-mon[29469]: Updating smithi152:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:36:48.107 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:48.107 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:48.107 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:48.107 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:48.108 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:36:48.139 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T01:36:48.373 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:36:49.187 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:48 smithi078 ceph-mon[29469]: Added host smithi196 2024-04-25T01:36:49.187 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:48 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:49.188 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:48 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:49.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.965+0000 7fe29d693700 1 -- 172.21.15.78:0/3039715499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe298100da0 msgr2=0x7fe298101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:49.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.965+0000 7fe29d693700 1 --2- 172.21.15.78:0/3039715499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe298100da0 0x7fe298101180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe288009910 tx=0x7fe288009c20 comp rx=0 tx=0).stop 2024-04-25T01:36:49.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.965+0000 7fe29d693700 1 -- 172.21.15.78:0/3039715499 shutdown_connections 2024-04-25T01:36:49.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.965+0000 7fe29d693700 1 --2- 172.21.15.78:0/3039715499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe298100da0 0x7fe298101180 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:49.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.965+0000 7fe29d693700 1 -- 172.21.15.78:0/3039715499 >> 172.21.15.78:0/3039715499 conn(0x7fe2980fc650 msgr2=0x7fe2980fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:49.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.965+0000 7fe29d693700 1 -- 172.21.15.78:0/3039715499 shutdown_connections 2024-04-25T01:36:49.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.965+0000 7fe29d693700 1 -- 172.21.15.78:0/3039715499 wait complete. 2024-04-25T01:36:49.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.966+0000 7fe29d693700 1 Processor -- start 2024-04-25T01:36:49.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.966+0000 7fe29d693700 1 -- start start 2024-04-25T01:36:49.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.966+0000 7fe29d693700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe298100da0 0x7fe298196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:49.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.966+0000 7fe29d693700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe29810b170 con 0x7fe298100da0 2024-04-25T01:36:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.966+0000 7fe296ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe298100da0 0x7fe298196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.966+0000 7fe296ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe298100da0 0x7fe298196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45606/0 (socket says 172.21.15.78:45606) 2024-04-25T01:36:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.966+0000 7fe296ffd700 1 -- 172.21.15.78:0/1008242011 learned_addr learned my addr 172.21.15.78:0/1008242011 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.967+0000 7fe296ffd700 1 -- 172.21.15.78:0/1008242011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe298196c50 con 0x7fe298100da0 2024-04-25T01:36:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.967+0000 7fe296ffd700 1 --2- 172.21.15.78:0/1008242011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe298100da0 0x7fe298196710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe288005850 tx=0x7fe288005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.967+0000 7fe27ffff700 1 -- 172.21.15.78:0/1008242011 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe288028070 con 0x7fe298100da0 2024-04-25T01:36:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.967+0000 7fe29d693700 1 -- 172.21.15.78:0/1008242011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe298196ee0 con 0x7fe298100da0 2024-04-25T01:36:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.967+0000 7fe27ffff700 1 -- 172.21.15.78:0/1008242011 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe28800bbd0 con 0x7fe298100da0 2024-04-25T01:36:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.967+0000 7fe27ffff700 1 -- 172.21.15.78:0/1008242011 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe288003970 con 0x7fe298100da0 2024-04-25T01:36:49.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.967+0000 7fe29d693700 1 -- 172.21.15.78:0/1008242011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe29819afe0 con 0x7fe298100da0 2024-04-25T01:36:49.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.968+0000 7fe27ffff700 1 -- 172.21.15.78:0/1008242011 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe28802b070 con 0x7fe298100da0 2024-04-25T01:36:49.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.968+0000 7fe29d693700 1 -- 172.21.15.78:0/1008242011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe298102280 con 0x7fe298100da0 2024-04-25T01:36:49.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.968+0000 7fe27ffff700 1 --2- 172.21.15.78:0/1008242011 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe2800390f0 0x7fe28003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:49.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.968+0000 7fe27ffff700 1 -- 172.21.15.78:0/1008242011 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe288020070 con 0x7fe298100da0 2024-04-25T01:36:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.968+0000 7fe2967fc700 1 --2- 172.21.15.78:0/1008242011 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe2800390f0 0x7fe28003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.969+0000 7fe2967fc700 1 --2- 172.21.15.78:0/1008242011 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe2800390f0 0x7fe28003b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe28c006fd0 tx=0x7fe28c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:49.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:49.973+0000 7fe27ffff700 1 -- 172.21.15.78:0/1008242011 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe288013070 con 0x7fe298100da0 2024-04-25T01:36:49.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:49 smithi078 ceph-mon[29469]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:36:50.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:50.180+0000 7fe29d693700 1 -- 172.21.15.78:0/1008242011 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe298197a70 con 0x7fe2800390f0 2024-04-25T01:36:50.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:50.181+0000 7fe27ffff700 1 -- 172.21.15.78:0/1008242011 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fe298197a70 con 0x7fe2800390f0 2024-04-25T01:36:50.157 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:36:50.157 INFO:teuthology.orchestra.run.smithi078.stdout:[{"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.152", "hostname": "smithi152", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-04-25T01:36:50.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:50.183+0000 7fe29d693700 1 -- 172.21.15.78:0/1008242011 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe2800390f0 msgr2=0x7fe28003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:50.183+0000 7fe29d693700 1 --2- 172.21.15.78:0/1008242011 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe2800390f0 0x7fe28003b5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe28c006fd0 tx=0x7fe28c006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:50.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:50.183+0000 7fe29d693700 1 -- 172.21.15.78:0/1008242011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe298100da0 msgr2=0x7fe298196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:50.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:50.183+0000 7fe29d693700 1 --2- 172.21.15.78:0/1008242011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe298100da0 0x7fe298196710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe288005850 tx=0x7fe288005030 comp rx=0 tx=0).stop 2024-04-25T01:36:50.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:50.183+0000 7fe29d693700 1 -- 172.21.15.78:0/1008242011 shutdown_connections 2024-04-25T01:36:50.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:50.183+0000 7fe29d693700 1 --2- 172.21.15.78:0/1008242011 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe2800390f0 0x7fe28003b5b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:50.183+0000 7fe29d693700 1 --2- 172.21.15.78:0/1008242011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe298100da0 0x7fe298196710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:50.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:50.183+0000 7fe29d693700 1 -- 172.21.15.78:0/1008242011 >> 172.21.15.78:0/1008242011 conn(0x7fe2980fc650 msgr2=0x7fe298103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:50.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:50.183+0000 7fe29d693700 1 -- 172.21.15.78:0/1008242011 shutdown_connections 2024-04-25T01:36:50.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:50.183+0000 7fe29d693700 1 -- 172.21.15.78:0/1008242011 wait complete. 2024-04-25T01:36:50.809 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T01:36:50.810 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T01:36:51.044 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:36:51.396 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:51.397 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:51 smithi078 ceph-mon[29469]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:36:51.397 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:51 smithi078 ceph-mon[29469]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:36:52.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.528+0000 7efe719cd700 1 -- 172.21.15.78:0/470193258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c100da0 msgr2=0x7efe6c101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:52.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.528+0000 7efe719cd700 1 --2- 172.21.15.78:0/470193258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c100da0 0x7efe6c101180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009910 tx=0x7efe5c009c20 comp rx=0 tx=0).stop 2024-04-25T01:36:52.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.528+0000 7efe719cd700 1 -- 172.21.15.78:0/470193258 shutdown_connections 2024-04-25T01:36:52.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.528+0000 7efe719cd700 1 --2- 172.21.15.78:0/470193258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c100da0 0x7efe6c101180 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:52.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.528+0000 7efe719cd700 1 -- 172.21.15.78:0/470193258 >> 172.21.15.78:0/470193258 conn(0x7efe6c0fc630 msgr2=0x7efe6c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:52.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.528+0000 7efe719cd700 1 -- 172.21.15.78:0/470193258 shutdown_connections 2024-04-25T01:36:52.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.528+0000 7efe719cd700 1 -- 172.21.15.78:0/470193258 wait complete. 2024-04-25T01:36:52.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.529+0000 7efe719cd700 1 Processor -- start 2024-04-25T01:36:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.529+0000 7efe719cd700 1 -- start start 2024-04-25T01:36:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.529+0000 7efe719cd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c100da0 0x7efe6c102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.529+0000 7efe719cd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c10b170 con 0x7efe6c100da0 2024-04-25T01:36:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.529+0000 7efe6affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c100da0 0x7efe6c102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.529+0000 7efe6affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c100da0 0x7efe6c102840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45628/0 (socket says 172.21.15.78:45628) 2024-04-25T01:36:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.529+0000 7efe6affd700 1 -- 172.21.15.78:0/3541134326 learned_addr learned my addr 172.21.15.78:0/3541134326 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:36:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.529+0000 7efe6affd700 1 -- 172.21.15.78:0/3541134326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c104730 con 0x7efe6c100da0 2024-04-25T01:36:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.529+0000 7efe6affd700 1 --2- 172.21.15.78:0/3541134326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c100da0 0x7efe6c102840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe5c005850 tx=0x7efe5c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.530+0000 7efe709cb700 1 -- 172.21.15.78:0/3541134326 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe5c026070 con 0x7efe6c100da0 2024-04-25T01:36:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.530+0000 7efe709cb700 1 -- 172.21.15.78:0/3541134326 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe5c00bc30 con 0x7efe6c100da0 2024-04-25T01:36:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.530+0000 7efe719cd700 1 -- 172.21.15.78:0/3541134326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6c102de0 con 0x7efe6c100da0 2024-04-25T01:36:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.530+0000 7efe709cb700 1 -- 172.21.15.78:0/3541134326 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe5c003a20 con 0x7efe6c100da0 2024-04-25T01:36:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.530+0000 7efe719cd700 1 -- 172.21.15.78:0/3541134326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6c1032a0 con 0x7efe6c100da0 2024-04-25T01:36:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.531+0000 7efe709cb700 1 -- 172.21.15.78:0/3541134326 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7efe5c029070 con 0x7efe6c100da0 2024-04-25T01:36:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.531+0000 7efe719cd700 1 -- 172.21.15.78:0/3541134326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe6c102280 con 0x7efe6c100da0 2024-04-25T01:36:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.531+0000 7efe709cb700 1 --2- 172.21.15.78:0/3541134326 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efe580390f0 0x7efe5803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.531+0000 7efe709cb700 1 -- 172.21.15.78:0/3541134326 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efe5c01e070 con 0x7efe6c100da0 2024-04-25T01:36:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.531+0000 7efe627fc700 1 --2- 172.21.15.78:0/3541134326 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efe580390f0 0x7efe5803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.531+0000 7efe627fc700 1 --2- 172.21.15.78:0/3541134326 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efe580390f0 0x7efe5803b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efe54006fd0 tx=0x7efe54006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.536+0000 7efe709cb700 1 -- 172.21.15.78:0/3541134326 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe5c013070 con 0x7efe6c100da0 2024-04-25T01:36:52.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.736+0000 7efe719cd700 1 -- 172.21.15.78:0/3541134326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7efe6c103d70 con 0x7efe6c100da0 2024-04-25T01:36:52.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.787+0000 7efe709cb700 1 -- 172.21.15.78:0/3541134326 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7efe5c030020 con 0x7efe6c100da0 2024-04-25T01:36:52.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.790+0000 7efe719cd700 1 -- 172.21.15.78:0/3541134326 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efe580390f0 msgr2=0x7efe5803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:52.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.790+0000 7efe719cd700 1 --2- 172.21.15.78:0/3541134326 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efe580390f0 0x7efe5803b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efe54006fd0 tx=0x7efe54006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:52.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.790+0000 7efe719cd700 1 -- 172.21.15.78:0/3541134326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c100da0 msgr2=0x7efe6c102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:52.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.790+0000 7efe719cd700 1 --2- 172.21.15.78:0/3541134326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c100da0 0x7efe6c102840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe5c005850 tx=0x7efe5c005030 comp rx=0 tx=0).stop 2024-04-25T01:36:52.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.790+0000 7efe719cd700 1 -- 172.21.15.78:0/3541134326 shutdown_connections 2024-04-25T01:36:52.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.790+0000 7efe719cd700 1 --2- 172.21.15.78:0/3541134326 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efe580390f0 0x7efe5803b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:52.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.790+0000 7efe719cd700 1 --2- 172.21.15.78:0/3541134326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe6c100da0 0x7efe6c102840 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:52.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.790+0000 7efe719cd700 1 -- 172.21.15.78:0/3541134326 >> 172.21.15.78:0/3541134326 conn(0x7efe6c0fc630 msgr2=0x7efe6c06a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:52.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.790+0000 7efe719cd700 1 -- 172.21.15.78:0/3541134326 shutdown_connections 2024-04-25T01:36:52.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:36:52.790+0000 7efe719cd700 1 -- 172.21.15.78:0/3541134326 wait complete. 2024-04-25T01:36:52.768 INFO:teuthology.orchestra.run.smithi078.stderr:adjusted tunables profile to default 2024-04-25T01:36:52.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:52 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3541134326' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T01:36:53.391 INFO:tasks.cephadm:Adding mon.a on smithi078 2024-04-25T01:36:53.391 INFO:tasks.cephadm:Adding mon.b on smithi152 2024-04-25T01:36:53.391 INFO:tasks.cephadm:Adding mon.c on smithi196 2024-04-25T01:36:53.392 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi078:172.21.15.78=a;smithi152:172.21.15.152=b;smithi196:172.21.15.196=c' 2024-04-25T01:36:53.621 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:36:54.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:53 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3541134326' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T01:36:54.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:53 smithi078 ceph-mon[29469]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:36:54.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:53 smithi078 ceph-mon[29469]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:36:54.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:54.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:54.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:54.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:54.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:54.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:36:54.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:36:55.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.147+0000 7f96a1626700 1 -- 172.21.15.196:0/3994438630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c100da0 msgr2=0x7f969c101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:55.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.147+0000 7f96a1626700 1 --2- 172.21.15.196:0/3994438630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c100da0 0x7f969c101180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9690009910 tx=0x7f9690009c20 comp rx=0 tx=0).stop 2024-04-25T01:36:55.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.147+0000 7f96a1626700 1 -- 172.21.15.196:0/3994438630 shutdown_connections 2024-04-25T01:36:55.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.147+0000 7f96a1626700 1 --2- 172.21.15.196:0/3994438630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c100da0 0x7f969c101180 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:55.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.147+0000 7f96a1626700 1 -- 172.21.15.196:0/3994438630 >> 172.21.15.196:0/3994438630 conn(0x7f969c0fc630 msgr2=0x7f969c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:55.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.147+0000 7f96a1626700 1 -- 172.21.15.196:0/3994438630 shutdown_connections 2024-04-25T01:36:55.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.147+0000 7f96a1626700 1 -- 172.21.15.196:0/3994438630 wait complete. 2024-04-25T01:36:55.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.147+0000 7f96a1626700 1 Processor -- start 2024-04-25T01:36:55.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.148+0000 7f96a1626700 1 -- start start 2024-04-25T01:36:55.124 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.148+0000 7f96a1626700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c100da0 0x7f969c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:55.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.148+0000 7f96a1626700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f969c10b170 con 0x7f969c100da0 2024-04-25T01:36:55.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.148+0000 7f969affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c100da0 0x7f969c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:55.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.148+0000 7f969affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c100da0 0x7f969c198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.196:39212/0 (socket says 172.21.15.196:39212) 2024-04-25T01:36:55.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.148+0000 7f969affd700 1 -- 172.21.15.196:0/2119600356 learned_addr learned my addr 172.21.15.196:0/2119600356 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T01:36:55.125 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.148+0000 7f969affd700 1 -- 172.21.15.196:0/2119600356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969c198b60 con 0x7f969c100da0 2024-04-25T01:36:55.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.149+0000 7f969affd700 1 --2- 172.21.15.196:0/2119600356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c100da0 0x7f969c198620 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9690005850 tx=0x7f9690005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:55.126 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.149+0000 7f9683fff700 1 -- 172.21.15.196:0/2119600356 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9690026070 con 0x7f969c100da0 2024-04-25T01:36:55.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.149+0000 7f96a1626700 1 -- 172.21.15.196:0/2119600356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969c198e50 con 0x7f969c100da0 2024-04-25T01:36:55.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.149+0000 7f9683fff700 1 -- 172.21.15.196:0/2119600356 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f969000bbd0 con 0x7f969c100da0 2024-04-25T01:36:55.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.149+0000 7f9683fff700 1 -- 172.21.15.196:0/2119600356 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9690003970 con 0x7f969c100da0 2024-04-25T01:36:55.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.149+0000 7f96a1626700 1 -- 172.21.15.196:0/2119600356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f969c199310 con 0x7f969c100da0 2024-04-25T01:36:55.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.150+0000 7f9683fff700 1 -- 172.21.15.196:0/2119600356 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9690029070 con 0x7f969c100da0 2024-04-25T01:36:55.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.150+0000 7f96a1626700 1 -- 172.21.15.196:0/2119600356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f969c102280 con 0x7f969c100da0 2024-04-25T01:36:55.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.150+0000 7f9683fff700 1 --2- 172.21.15.196:0/2119600356 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96840390f0 0x7f968403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:55.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.150+0000 7f9683fff700 1 -- 172.21.15.196:0/2119600356 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f969001e070 con 0x7f969c100da0 2024-04-25T01:36:55.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.150+0000 7f969a7fc700 1 --2- 172.21.15.196:0/2119600356 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96840390f0 0x7f968403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:55.127 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.151+0000 7f969a7fc700 1 --2- 172.21.15.196:0/2119600356 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96840390f0 0x7f968403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f968c006fd0 tx=0x7f968c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:55.130 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.155+0000 7f9683fff700 1 -- 172.21.15.196:0/2119600356 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9690013070 con 0x7f969c100da0 2024-04-25T01:36:55.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:54 smithi078 ceph-mon[29469]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-25T01:36:55.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:54 smithi078 ceph-mon[29469]: Updating smithi196:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:36:55.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:54 smithi078 ceph-mon[29469]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:36:55.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:54 smithi078 ceph-mon[29469]: Updating smithi196:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:36:55.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:55.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:55.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:55.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.359+0000 7f96a1626700 1 -- 172.21.15.196:0/2119600356 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi152:172.21.15.152=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}) -- 0x7f969c06a690 con 0x7f96840390f0 2024-04-25T01:36:55.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.366+0000 7f9683fff700 1 -- 172.21.15.196:0/2119600356 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f969c06a690 con 0x7f96840390f0 2024-04-25T01:36:55.341 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-04-25T01:36:55.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.368+0000 7f96a1626700 1 -- 172.21.15.196:0/2119600356 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96840390f0 msgr2=0x7f968403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:55.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.368+0000 7f96a1626700 1 --2- 172.21.15.196:0/2119600356 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96840390f0 0x7f968403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f968c006fd0 tx=0x7f968c006e40 comp rx=0 tx=0).stop 2024-04-25T01:36:55.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.368+0000 7f96a1626700 1 -- 172.21.15.196:0/2119600356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c100da0 msgr2=0x7f969c198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:55.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.368+0000 7f96a1626700 1 --2- 172.21.15.196:0/2119600356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c100da0 0x7f969c198620 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9690005850 tx=0x7f9690005030 comp rx=0 tx=0).stop 2024-04-25T01:36:55.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.368+0000 7f96a1626700 1 -- 172.21.15.196:0/2119600356 shutdown_connections 2024-04-25T01:36:55.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.368+0000 7f96a1626700 1 --2- 172.21.15.196:0/2119600356 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96840390f0 0x7f968403b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:55.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.368+0000 7f96a1626700 1 --2- 172.21.15.196:0/2119600356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c100da0 0x7f969c198620 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:55.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.368+0000 7f96a1626700 1 -- 172.21.15.196:0/2119600356 >> 172.21.15.196:0/2119600356 conn(0x7f969c0fc630 msgr2=0x7f969c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:55.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.368+0000 7f96a1626700 1 -- 172.21.15.196:0/2119600356 shutdown_connections 2024-04-25T01:36:55.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:55.368+0000 7f96a1626700 1 -- 172.21.15.196:0/2119600356 wait complete. 2024-04-25T01:36:55.913 DEBUG:teuthology.orchestra.run.smithi152:mon.b> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T01:36:55.916 DEBUG:teuthology.orchestra.run.smithi196:mon.c> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T01:36:55.918 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T01:36:55.918 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:36:56.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:55 smithi078 ceph-mon[29469]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:36:56.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:56.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:36:56.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:36:56.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:36:56.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:36:56.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:36:56.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:36:57.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:56 smithi078 ceph-mon[29469]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi152:172.21.15.152=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:57.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:56 smithi078 ceph-mon[29469]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi152:172.21.15.152=b;smithi196:172.21.15.196=c;count:3 2024-04-25T01:36:57.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:56 smithi078 ceph-mon[29469]: Deploying daemon mon.c on smithi196 2024-04-25T01:36:57.937 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:36:58.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:57 smithi078 ceph-mon[29469]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:36:59.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:36:58 smithi078 ceph-mon[29469]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:36:59.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.803+0000 7f4e1131f700 1 -- 172.21.15.196:0/2390945156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e0c101fb0 msgr2=0x7f4e0c102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:59.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.803+0000 7f4e1131f700 1 --2- 172.21.15.196:0/2390945156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e0c101fb0 0x7f4e0c102390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400b0a0 tx=0x7f4e0400b3b0 comp rx=0 tx=0).stop 2024-04-25T01:36:59.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.803+0000 7f4e1131f700 1 -- 172.21.15.196:0/2390945156 shutdown_connections 2024-04-25T01:36:59.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.803+0000 7f4e1131f700 1 --2- 172.21.15.196:0/2390945156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e0c101fb0 0x7f4e0c102390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:59.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.803+0000 7f4e1131f700 1 -- 172.21.15.196:0/2390945156 >> 172.21.15.196:0/2390945156 conn(0x7f4e0c0fd840 msgr2=0x7f4e0c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:59.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.803+0000 7f4e1131f700 1 -- 172.21.15.196:0/2390945156 shutdown_connections 2024-04-25T01:36:59.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.803+0000 7f4e1131f700 1 -- 172.21.15.196:0/2390945156 wait complete. 2024-04-25T01:36:59.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.803+0000 7f4e1131f700 1 Processor -- start 2024-04-25T01:36:59.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.803+0000 7f4e1131f700 1 -- start start 2024-04-25T01:36:59.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.804+0000 7f4e1131f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e0c101fb0 0x7f4e0c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:59.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.804+0000 7f4e1131f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c10c380 con 0x7f4e0c101fb0 2024-04-25T01:36:59.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.804+0000 7f4e0affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e0c101fb0 0x7f4e0c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:59.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.804+0000 7f4e0affd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e0c101fb0 0x7f4e0c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.196:36672/0 (socket says 172.21.15.196:36672) 2024-04-25T01:36:59.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.804+0000 7f4e0affd700 1 -- 172.21.15.196:0/3041700251 learned_addr learned my addr 172.21.15.196:0/3041700251 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T01:36:59.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.804+0000 7f4e0affd700 1 -- 172.21.15.196:0/3041700251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0c19db80 con 0x7f4e0c101fb0 2024-04-25T01:36:59.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.805+0000 7f4e0affd700 1 --2- 172.21.15.196:0/3041700251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e0c101fb0 0x7f4e0c19d640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4e04015040 tx=0x7f4e04003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:59.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.805+0000 7f4e03fff700 1 -- 172.21.15.196:0/3041700251 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e04017040 con 0x7f4e0c101fb0 2024-04-25T01:36:59.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.805+0000 7f4e1131f700 1 -- 172.21.15.196:0/3041700251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0c19de10 con 0x7f4e0c101fb0 2024-04-25T01:36:59.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.805+0000 7f4e03fff700 1 -- 172.21.15.196:0/3041700251 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e04022070 con 0x7f4e0c101fb0 2024-04-25T01:36:59.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.805+0000 7f4e03fff700 1 -- 172.21.15.196:0/3041700251 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e0401d5d0 con 0x7f4e0c101fb0 2024-04-25T01:36:59.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.805+0000 7f4e1131f700 1 -- 172.21.15.196:0/3041700251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e0c197710 con 0x7f4e0c101fb0 2024-04-25T01:36:59.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.806+0000 7f4e03fff700 1 -- 172.21.15.196:0/3041700251 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4e04004000 con 0x7f4e0c101fb0 2024-04-25T01:36:59.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.806+0000 7f4e1131f700 1 -- 172.21.15.196:0/3041700251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e0c103490 con 0x7f4e0c101fb0 2024-04-25T01:36:59.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.806+0000 7f4e03fff700 1 --2- 172.21.15.196:0/3041700251 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4df80390f0 0x7f4df803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:59.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.806+0000 7f4e03fff700 1 -- 172.21.15.196:0/3041700251 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4e0400f050 con 0x7f4e0c101fb0 2024-04-25T01:36:59.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.806+0000 7f4e0a7fc700 1 --2- 172.21.15.196:0/3041700251 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4df80390f0 0x7f4df803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:59.783 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.807+0000 7f4e0a7fc700 1 --2- 172.21.15.196:0/3041700251 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4df80390f0 0x7f4df803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4df4006fd0 tx=0x7f4df4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:59.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:36:59.812+0000 7f4e03fff700 1 -- 172.21.15.196:0/3041700251 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4e04007e20 con 0x7f4e0c101fb0 2024-04-25T01:37:00.081 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:00.105+0000 7f4e1131f700 1 -- 172.21.15.196:0/3041700251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4e0c063890 con 0x7f4e0c101fb0 2024-04-25T01:37:00.082 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:00.106+0000 7f4e03fff700 1 -- 172.21.15.196:0/3041700251 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f4e04004440 con 0x7f4e0c101fb0 2024-04-25T01:37:00.082 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:37:00.082 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","modified":"2024-04-25T01:35:19.558079Z","created":"2024-04-25T01:35:19.558079Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T01:37:00.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:00.108+0000 7f4e1131f700 1 -- 172.21.15.196:0/3041700251 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4df80390f0 msgr2=0x7f4df803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:00.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:00.108+0000 7f4e1131f700 1 --2- 172.21.15.196:0/3041700251 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4df80390f0 0x7f4df803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4df4006fd0 tx=0x7f4df4006e40 comp rx=0 tx=0).stop 2024-04-25T01:37:00.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:00.109+0000 7f4e1131f700 1 -- 172.21.15.196:0/3041700251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e0c101fb0 msgr2=0x7f4e0c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:00.084 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:00.109+0000 7f4e1131f700 1 --2- 172.21.15.196:0/3041700251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e0c101fb0 0x7f4e0c19d640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4e04015040 tx=0x7f4e04003d50 comp rx=0 tx=0).stop 2024-04-25T01:37:00.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:00.109+0000 7f4e1131f700 1 -- 172.21.15.196:0/3041700251 shutdown_connections 2024-04-25T01:37:00.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:00.109+0000 7f4e1131f700 1 --2- 172.21.15.196:0/3041700251 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4df80390f0 0x7f4df803b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:00.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:00.109+0000 7f4e1131f700 1 --2- 172.21.15.196:0/3041700251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e0c101fb0 0x7f4e0c19d640 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:00.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:00.109+0000 7f4e1131f700 1 -- 172.21.15.196:0/3041700251 >> 172.21.15.196:0/3041700251 conn(0x7f4e0c0fd840 msgr2=0x7f4e0c103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:00.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:00.109+0000 7f4e1131f700 1 -- 172.21.15.196:0/3041700251 shutdown_connections 2024-04-25T01:37:00.085 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:00.109+0000 7f4e1131f700 1 -- 172.21.15.196:0/3041700251 wait complete. 2024-04-25T01:37:00.085 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-04-25T01:37:00.491 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:00 smithi078 ceph-mon[29469]: from='client.? 172.21.15.196:0/3041700251' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:37:00.674 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:00 smithi196 systemd[1]: Started Ceph mon.c for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:37:02.022 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T01:37:02.022 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:37:02.263 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:37:05.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:37:05.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:05.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: mon.a calling monitor election 2024-04-25T01:37:05.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:05.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:05.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:05.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: mon.c calling monitor election 2024-04-25T01:37:05.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:05.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:05.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:05.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:05.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:05.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T01:37:05.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: monmap epoch 2 2024-04-25T01:37:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: fsid df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:37:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: last_changed 2024-04-25T01:37:00.489330+0000 2024-04-25T01:37:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: created 2024-04-25T01:35:19.558079+0000 2024-04-25T01:37:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: min_mon_release 19 (squid) 2024-04-25T01:37:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: election_strategy: 1 2024-04-25T01:37:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T01:37:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-25T01:37:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: fsmap 2024-04-25T01:37:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:37:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: mgrmap e13: x(active, since 48s) 2024-04-25T01:37:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: overall HEALTH_OK 2024-04-25T01:37:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:05.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:05.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:37:05.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:05.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:37:05.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:05.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: mon.a calling monitor election 2024-04-25T01:37:05.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:05.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:05.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:05.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: mon.c calling monitor election 2024-04-25T01:37:05.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T01:37:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: monmap epoch 2 2024-04-25T01:37:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: fsid df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:37:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: last_changed 2024-04-25T01:37:00.489330+0000 2024-04-25T01:37:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: created 2024-04-25T01:35:19.558079+0000 2024-04-25T01:37:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: min_mon_release 19 (squid) 2024-04-25T01:37:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: election_strategy: 1 2024-04-25T01:37:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T01:37:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-25T01:37:05.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: fsmap 2024-04-25T01:37:05.900 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:37:05.900 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: mgrmap e13: x(active, since 48s) 2024-04-25T01:37:05.900 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: overall HEALTH_OK 2024-04-25T01:37:05.900 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:05.900 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:05.900 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:05.900 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:37:05.900 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:06.669 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.693+0000 7f4dd73db700 1 -- 172.21.15.196:0/739623118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dbc0035c0 msgr2=0x7f4dbc005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:06.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.693+0000 7f4dd73db700 1 --2- 172.21.15.196:0/739623118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dbc0035c0 0x7f4dbc005a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0079470 tx=0x7f4dc0010ec0 comp rx=0 tx=0).stop 2024-04-25T01:37:06.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.693+0000 7f4dd73db700 1 -- 172.21.15.196:0/739623118 shutdown_connections 2024-04-25T01:37:06.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.693+0000 7f4dd73db700 1 --2- 172.21.15.196:0/739623118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dbc0035c0 0x7f4dbc005a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:06.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.693+0000 7f4dd73db700 1 -- 172.21.15.196:0/739623118 >> 172.21.15.196:0/739623118 conn(0x7f4dd00772e0 msgr2=0x7f4dd00776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:06.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.694+0000 7f4dd73db700 1 -- 172.21.15.196:0/739623118 shutdown_connections 2024-04-25T01:37:06.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.694+0000 7f4dd73db700 1 -- 172.21.15.196:0/739623118 wait complete. 2024-04-25T01:37:06.671 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd73db700 1 Processor -- start 2024-04-25T01:37:06.671 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd73db700 1 -- start start 2024-04-25T01:37:06.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd73db700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dbc0035c0 0x7f4dd019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:06.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd73db700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd019dbe0 0x7f4dd0197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:06.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd73db700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd0107be0 con 0x7f4dbc0035c0 2024-04-25T01:37:06.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd73db700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd0107a60 con 0x7f4dd019dbe0 2024-04-25T01:37:06.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd4976700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd019dbe0 0x7f4dd0197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:06.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd5177700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dbc0035c0 0x7f4dd019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:06.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd4976700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd019dbe0 0x7f4dd0197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:49074/0 (socket says 172.21.15.196:49074) 2024-04-25T01:37:06.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd5177700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dbc0035c0 0x7f4dd019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.196:36726/0 (socket says 172.21.15.196:36726) 2024-04-25T01:37:06.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd4976700 1 -- 172.21.15.196:0/3486825739 learned_addr learned my addr 172.21.15.196:0/3486825739 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T01:37:06.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd4976700 1 -- 172.21.15.196:0/3486825739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd019dbe0 msgr2=0x7f4dd0197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T01:37:06.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd4976700 1 -- 172.21.15.196:0/3486825739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd019dbe0 msgr2=0x7f4dd0197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:37:06.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd4976700 1 --2- 172.21.15.196:0/3486825739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd019dbe0 0x7f4dd0197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:37:06.672 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd4976700 1 --2- 172.21.15.196:0/3486825739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd019dbe0 0x7f4dd0197770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:06.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd5177700 1 -- 172.21.15.196:0/3486825739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd019dbe0 msgr2=0x7f4dd0197770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:37:06.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd5177700 1 --2- 172.21.15.196:0/3486825739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd019dbe0 0x7f4dd0197770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:06.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.695+0000 7f4dd5177700 1 -- 172.21.15.196:0/3486825739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd019e160 con 0x7f4dbc0035c0 2024-04-25T01:37:06.673 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.696+0000 7f4dd5177700 1 --2- 172.21.15.196:0/3486825739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dbc0035c0 0x7f4dd019d6a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4dc001d040 tx=0x7f4dc00047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:06.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.696+0000 7f4dc67fc700 1 -- 172.21.15.196:0/3486825739 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4dc00041d0 con 0x7f4dbc0035c0 2024-04-25T01:37:06.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.696+0000 7f4dc67fc700 1 -- 172.21.15.196:0/3486825739 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc0012d50 con 0x7f4dbc0035c0 2024-04-25T01:37:06.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.696+0000 7f4dc67fc700 1 -- 172.21.15.196:0/3486825739 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4dc001bd90 con 0x7f4dbc0035c0 2024-04-25T01:37:06.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.696+0000 7f4dd73db700 1 -- 172.21.15.196:0/3486825739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd0197d10 con 0x7f4dbc0035c0 2024-04-25T01:37:06.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.696+0000 7f4dd73db700 1 -- 172.21.15.196:0/3486825739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd01982a0 con 0x7f4dbc0035c0 2024-04-25T01:37:06.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.697+0000 7f4dc67fc700 1 -- 172.21.15.196:0/3486825739 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4dc0003680 con 0x7f4dbc0035c0 2024-04-25T01:37:06.674 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.697+0000 7f4dc67fc700 1 --2- 172.21.15.196:0/3486825739 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4db4038e90 0x7f4db403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:06.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.697+0000 7f4dc67fc700 1 -- 172.21.15.196:0/3486825739 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4dc0018070 con 0x7f4dbc0035c0 2024-04-25T01:37:06.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.697+0000 7f4dd73db700 1 -- 172.21.15.196:0/3486825739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd0101b10 con 0x7f4dbc0035c0 2024-04-25T01:37:06.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.698+0000 7f4dd4976700 1 --2- 172.21.15.196:0/3486825739 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4db4038e90 0x7f4db403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:06.675 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.698+0000 7f4dd4976700 1 --2- 172.21.15.196:0/3486825739 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4db4038e90 0x7f4db403b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009180 tx=0x7f4dcc008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:06.678 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.703+0000 7f4dc67fc700 1 -- 172.21.15.196:0/3486825739 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4dc004a070 con 0x7f4dbc0035c0 2024-04-25T01:37:06.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:06 smithi196 ceph-mon[32978]: Deploying daemon mon.b on smithi152 2024-04-25T01:37:06.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:06 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:06.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.986+0000 7f4dd73db700 1 -- 172.21.15.196:0/3486825739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4dd0198ca0 con 0x7f4dbc0035c0 2024-04-25T01:37:06.962 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.986+0000 7f4dc67fc700 1 -- 172.21.15.196:0/3486825739 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f4dc000ec60 con 0x7f4dbc0035c0 2024-04-25T01:37:06.963 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:37:06.963 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","modified":"2024-04-25T01:37:00.489330Z","created":"2024-04-25T01:35:19.558079Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T01:37:06.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.988+0000 7f4dd73db700 1 -- 172.21.15.196:0/3486825739 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4db4038e90 msgr2=0x7f4db403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:06.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.988+0000 7f4dd73db700 1 --2- 172.21.15.196:0/3486825739 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4db4038e90 0x7f4db403b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009180 tx=0x7f4dcc008040 comp rx=0 tx=0).stop 2024-04-25T01:37:06.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.988+0000 7f4dd73db700 1 -- 172.21.15.196:0/3486825739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dbc0035c0 msgr2=0x7f4dd019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:06.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.988+0000 7f4dd73db700 1 --2- 172.21.15.196:0/3486825739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dbc0035c0 0x7f4dd019d6a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4dc001d040 tx=0x7f4dc00047a0 comp rx=0 tx=0).stop 2024-04-25T01:37:06.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.988+0000 7f4dd73db700 1 -- 172.21.15.196:0/3486825739 shutdown_connections 2024-04-25T01:37:06.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.988+0000 7f4dd73db700 1 --2- 172.21.15.196:0/3486825739 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4db4038e90 0x7f4db403b350 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:06.965 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.988+0000 7f4dd73db700 1 --2- 172.21.15.196:0/3486825739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dbc0035c0 0x7f4dd019d6a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:06.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.989+0000 7f4dd73db700 1 --2- 172.21.15.196:0/3486825739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd019dbe0 0x7f4dd0197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:06.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.989+0000 7f4dd73db700 1 -- 172.21.15.196:0/3486825739 >> 172.21.15.196:0/3486825739 conn(0x7f4dd00772e0 msgr2=0x7f4dd00ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:06.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.989+0000 7f4dd73db700 1 -- 172.21.15.196:0/3486825739 shutdown_connections 2024-04-25T01:37:06.966 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:06.989+0000 7f4dd73db700 1 -- 172.21.15.196:0/3486825739 wait complete. 2024-04-25T01:37:06.966 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-04-25T01:37:06.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:06 smithi078 ceph-mon[29469]: Deploying daemon mon.b on smithi152 2024-04-25T01:37:06.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:06 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:07.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:07 smithi196 ceph-mon[32978]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:07.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:07 smithi196 ceph-mon[32978]: from='client.? 172.21.15.196:0/3486825739' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:37:07.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:07 smithi078 ceph-mon[29469]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:07.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:07 smithi078 ceph-mon[29469]: from='client.? 172.21.15.196:0/3486825739' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:37:08.575 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T01:37:08.575 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:37:08.812 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:37:09.783 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:09 smithi152 systemd[1]: Started Ceph mon.b for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:37:14.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.726+0000 7f0e45bf2700 1 -- 172.21.15.196:0/3492459868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e2c0035c0 msgr2=0x7f0e2c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:14.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.726+0000 7f0e45bf2700 1 --2- 172.21.15.196:0/3492459868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e2c0035c0 0x7f0e2c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e40100b70 tx=0x7f0e2800fe60 comp rx=0 tx=0).stop 2024-04-25T01:37:14.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.726+0000 7f0e45bf2700 1 -- 172.21.15.196:0/3492459868 shutdown_connections 2024-04-25T01:37:14.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.726+0000 7f0e45bf2700 1 --2- 172.21.15.196:0/3492459868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e2c0035c0 0x7f0e2c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:14.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.726+0000 7f0e45bf2700 1 -- 172.21.15.196:0/3492459868 >> 172.21.15.196:0/3492459868 conn(0x7f0e400fc730 msgr2=0x7f0e400feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:14.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.726+0000 7f0e45bf2700 1 -- 172.21.15.196:0/3492459868 shutdown_connections 2024-04-25T01:37:14.703 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.726+0000 7f0e45bf2700 1 -- 172.21.15.196:0/3492459868 wait complete. 2024-04-25T01:37:14.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.727+0000 7f0e45bf2700 1 Processor -- start 2024-04-25T01:37:14.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.727+0000 7f0e45bf2700 1 -- start start 2024-04-25T01:37:14.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e45bf2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e2c0035c0 0x7f0e40102940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:14.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e45bf2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e40102e80 0x7f0e40103300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:14.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e45bf2700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e401a3ba0 0x7f0e401a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:14.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e45bf2700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e40108c80 con 0x7f0e401a3ba0 2024-04-25T01:37:14.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e45bf2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0e40108b00 con 0x7f0e40102e80 2024-04-25T01:37:14.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e45bf2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e40108e00 con 0x7f0e2c0035c0 2024-04-25T01:37:14.705 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e3f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e2c0035c0 0x7f0e40102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:14.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e3f7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e2c0035c0 0x7f0e40102940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:50466/0 (socket says 172.21.15.196:50466) 2024-04-25T01:37:14.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e3f7fe700 1 -- 172.21.15.196:0/1125551942 learned_addr learned my addr 172.21.15.196:0/1125551942 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T01:37:14.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e3f7fe700 1 -- 172.21.15.196:0/1125551942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e2c0035c0 msgr2=0x7f0e40102940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T01:37:14.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e3f7fe700 1 -- 172.21.15.196:0/1125551942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e2c0035c0 msgr2=0x7f0e40102940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:37:14.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e3f7fe700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e2c0035c0 0x7f0e40102940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:37:14.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e3ffff700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e401a3ba0 0x7f0e401a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:14.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e3f7fe700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e2c0035c0 0x7f0e40102940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:14.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e3effd700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e40102e80 0x7f0e40103300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:14.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e3effd700 1 -- 172.21.15.196:0/1125551942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e40102e80 msgr2=0x7f0e40103300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T01:37:14.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e3effd700 1 -- 172.21.15.196:0/1125551942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e40102e80 msgr2=0x7f0e40103300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:37:14.706 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e3effd700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e40102e80 0x7f0e40103300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:37:14.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.728+0000 7f0e3effd700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e40102e80 0x7f0e40103300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:37:14.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.729+0000 7f0e3ffff700 1 -- 172.21.15.196:0/1125551942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e2c0035c0 msgr2=0x7f0e40102940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:37:14.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.729+0000 7f0e3ffff700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e2c0035c0 0x7f0e40102940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:14.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.729+0000 7f0e3ffff700 1 -- 172.21.15.196:0/1125551942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e40102e80 msgr2=0x7f0e40103300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:37:14.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.729+0000 7f0e3ffff700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e40102e80 0x7f0e40103300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:14.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.729+0000 7f0e3ffff700 1 -- 172.21.15.196:0/1125551942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e401a66c0 con 0x7f0e401a3ba0 2024-04-25T01:37:14.707 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.729+0000 7f0e3ffff700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e401a3ba0 0x7f0e401a5f90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800bd50 tx=0x7f0e3800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:14.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.729+0000 7f0e3cff9700 1 -- 172.21.15.196:0/1125551942 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38019070 con 0x7f0e401a3ba0 2024-04-25T01:37:14.709 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.729+0000 7f0e3cff9700 1 -- 172.21.15.196:0/1125551942 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e3800cd10 con 0x7f0e401a3ba0 2024-04-25T01:37:14.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.729+0000 7f0e45bf2700 1 -- 172.21.15.196:0/1125551942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e401a69b0 con 0x7f0e401a3ba0 2024-04-25T01:37:14.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.729+0000 7f0e3cff9700 1 -- 172.21.15.196:0/1125551942 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e380127e0 con 0x7f0e401a3ba0 2024-04-25T01:37:14.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.729+0000 7f0e45bf2700 1 -- 172.21.15.196:0/1125551942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e401a6f40 con 0x7f0e401a3ba0 2024-04-25T01:37:14.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.730+0000 7f0e3cff9700 1 -- 172.21.15.196:0/1125551942 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0e3801c070 con 0x7f0e401a3ba0 2024-04-25T01:37:14.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.731+0000 7f0e3cff9700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0e20039580 0x7f0e2003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:14.710 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.731+0000 7f0e3cff9700 1 -- 172.21.15.196:0/1125551942 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0e3804bb60 con 0x7f0e401a3ba0 2024-04-25T01:37:14.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.731+0000 7f0e45bf2700 1 -- 172.21.15.196:0/1125551942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e40063890 con 0x7f0e401a3ba0 2024-04-25T01:37:14.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.731+0000 7f0e3f7fe700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0e20039580 0x7f0e2003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:14.711 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.732+0000 7f0e3f7fe700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0e20039580 0x7f0e2003ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0e28001eb0 tx=0x7f0e28005de0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:14.713 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:14.737+0000 7f0e3cff9700 1 -- 172.21.15.196:0/1125551942 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0e38013c30 con 0x7f0e401a3ba0 2024-04-25T01:37:14.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:37:14.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:14.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: mon.a calling monitor election 2024-04-25T01:37:14.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: mon.c calling monitor election 2024-04-25T01:37:14.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:14.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:14.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:14.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:14.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:14.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T01:37:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: monmap epoch 3 2024-04-25T01:37:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: fsid df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:37:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: last_changed 2024-04-25T01:37:09.547590+0000 2024-04-25T01:37:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: created 2024-04-25T01:35:19.558079+0000 2024-04-25T01:37:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: min_mon_release 19 (squid) 2024-04-25T01:37:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: election_strategy: 1 2024-04-25T01:37:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T01:37:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-25T01:37:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: 2: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-04-25T01:37:14.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: fsmap 2024-04-25T01:37:14.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:37:14.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: mgrmap e13: x(active, since 57s) 2024-04-25T01:37:14.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: overall HEALTH_OK 2024-04-25T01:37:14.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:14.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:14.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:14.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:14.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:37:14.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:14.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: mon.a calling monitor election 2024-04-25T01:37:14.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: mon.c calling monitor election 2024-04-25T01:37:14.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:14.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:14.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:14.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:14.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:14.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:14.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:14.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:14.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T01:37:14.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: monmap epoch 3 2024-04-25T01:37:14.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: fsid df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:37:14.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: last_changed 2024-04-25T01:37:09.547590+0000 2024-04-25T01:37:14.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: created 2024-04-25T01:35:19.558079+0000 2024-04-25T01:37:14.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: min_mon_release 19 (squid) 2024-04-25T01:37:14.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: election_strategy: 1 2024-04-25T01:37:14.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T01:37:14.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-25T01:37:14.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: 2: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-04-25T01:37:14.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: fsmap 2024-04-25T01:37:14.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:37:14.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: mgrmap e13: x(active, since 57s) 2024-04-25T01:37:14.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: overall HEALTH_OK 2024-04-25T01:37:14.995 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:14.995 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:14.995 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:14.995 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:15.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:15.026+0000 7f0e45bf2700 1 -- 172.21.15.196:0/1125551942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0e4002db30 con 0x7f0e401a3ba0 2024-04-25T01:37:15.003 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:15.027+0000 7f0e3cff9700 1 -- 172.21.15.196:0/1125551942 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f0e3801d730 con 0x7f0e401a3ba0 2024-04-25T01:37:15.003 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T01:37:15.003 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","modified":"2024-04-25T01:37:09.547590Z","created":"2024-04-25T01:35:19.558079Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T01:37:15.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:15.029+0000 7f0e45bf2700 1 -- 172.21.15.196:0/1125551942 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0e20039580 msgr2=0x7f0e2003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:15.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:15.029+0000 7f0e45bf2700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0e20039580 0x7f0e2003ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0e28001eb0 tx=0x7f0e28005de0 comp rx=0 tx=0).stop 2024-04-25T01:37:15.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:15.029+0000 7f0e45bf2700 1 -- 172.21.15.196:0/1125551942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e401a3ba0 msgr2=0x7f0e401a5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:15.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:15.029+0000 7f0e45bf2700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e401a3ba0 0x7f0e401a5f90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800bd50 tx=0x7f0e3800be30 comp rx=0 tx=0).stop 2024-04-25T01:37:15.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:15.029+0000 7f0e45bf2700 1 -- 172.21.15.196:0/1125551942 shutdown_connections 2024-04-25T01:37:15.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:15.029+0000 7f0e45bf2700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0e20039580 0x7f0e2003ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:15.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:15.029+0000 7f0e45bf2700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e2c0035c0 0x7f0e40102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:15.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:15.029+0000 7f0e45bf2700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e401a3ba0 0x7f0e401a5f90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:15.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:15.029+0000 7f0e45bf2700 1 --2- 172.21.15.196:0/1125551942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e40102e80 0x7f0e40103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:15.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:15.029+0000 7f0e45bf2700 1 -- 172.21.15.196:0/1125551942 >> 172.21.15.196:0/1125551942 conn(0x7f0e400fc730 msgr2=0x7f0e4018eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:15.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:15.029+0000 7f0e45bf2700 1 -- 172.21.15.196:0/1125551942 shutdown_connections 2024-04-25T01:37:15.006 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:15.029+0000 7f0e45bf2700 1 -- 172.21.15.196:0/1125551942 wait complete. 2024-04-25T01:37:15.007 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-04-25T01:37:15.655 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T01:37:15.655 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T01:37:15.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:15.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:15 smithi196 ceph-mon[32978]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:15.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:15 smithi196 ceph-mon[32978]: from='client.? 172.21.15.196:0/1125551942' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:37:15.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:15.898 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:37:15.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:15.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:15 smithi078 ceph-mon[29469]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:15.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:15 smithi078 ceph-mon[29469]: from='client.? 172.21.15.196:0/1125551942' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:37:15.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:16.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:37:16.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:16.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: mon.a calling monitor election 2024-04-25T01:37:16.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: mon.c calling monitor election 2024-04-25T01:37:16.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:37:16.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:16.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:16.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:16.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:16.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:16.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:16.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:16.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T01:37:16.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: monmap epoch 3 2024-04-25T01:37:16.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: fsid df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:37:16.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: last_changed 2024-04-25T01:37:09.547590+0000 2024-04-25T01:37:16.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: created 2024-04-25T01:35:19.558079+0000 2024-04-25T01:37:16.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: min_mon_release 19 (squid) 2024-04-25T01:37:16.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: election_strategy: 1 2024-04-25T01:37:16.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T01:37:16.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-25T01:37:16.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: 2: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-04-25T01:37:16.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: fsmap 2024-04-25T01:37:16.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:37:16.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: mgrmap e13: x(active, since 57s) 2024-04-25T01:37:16.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: overall HEALTH_OK 2024-04-25T01:37:16.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:16.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:16.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:16.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:16.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:16.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:16.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='client.? 172.21.15.196:0/1125551942' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:37:16.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:17.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.541+0000 7fa60f2d9700 1 -- 172.21.15.78:0/3084395996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa608106c60 msgr2=0x7fa608107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:17.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.541+0000 7fa60f2d9700 1 --2- 172.21.15.78:0/3084395996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa608106c60 0x7fa608107040 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa604009910 tx=0x7fa604009c20 comp rx=0 tx=0).stop 2024-04-25T01:37:17.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.541+0000 7fa60f2d9700 1 -- 172.21.15.78:0/3084395996 shutdown_connections 2024-04-25T01:37:17.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.541+0000 7fa60f2d9700 1 --2- 172.21.15.78:0/3084395996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa608106c60 0x7fa608107040 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:17.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.541+0000 7fa60f2d9700 1 -- 172.21.15.78:0/3084395996 >> 172.21.15.78:0/3084395996 conn(0x7fa6080fb9a0 msgr2=0x7fa6080fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:17.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.541+0000 7fa60f2d9700 1 -- 172.21.15.78:0/3084395996 shutdown_connections 2024-04-25T01:37:17.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.541+0000 7fa60f2d9700 1 -- 172.21.15.78:0/3084395996 wait complete. 2024-04-25T01:37:17.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.542+0000 7fa60f2d9700 1 Processor -- start 2024-04-25T01:37:17.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.542+0000 7fa60f2d9700 1 -- start start 2024-04-25T01:37:17.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.542+0000 7fa60f2d9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa608106c60 0x7fa60819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:17.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.542+0000 7fa60f2d9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa60819dba0 0x7fa6081977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:17.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.542+0000 7fa60f2d9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa608197d20 0x7fa6081981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:17.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.542+0000 7fa60f2d9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa60810c260 con 0x7fa60819dba0 2024-04-25T01:37:17.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.542+0000 7fa60f2d9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa60810c0e0 con 0x7fa608106c60 2024-04-25T01:37:17.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.542+0000 7fa60f2d9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa60810c3e0 con 0x7fa608197d20 2024-04-25T01:37:17.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.542+0000 7fa60c874700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa60819dba0 0x7fa6081977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:17.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.542+0000 7fa60c874700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa60819dba0 0x7fa6081977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58294/0 (socket says 172.21.15.78:58294) 2024-04-25T01:37:17.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.542+0000 7fa60c874700 1 -- 172.21.15.78:0/2148708969 learned_addr learned my addr 172.21.15.78:0/2148708969 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:37:17.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.542+0000 7fa60d876700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa608197d20 0x7fa6081981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:17.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.543+0000 7fa60d075700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa608106c60 0x7fa60819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:17.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.543+0000 7fa60d876700 1 -- 172.21.15.78:0/2148708969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa608106c60 msgr2=0x7fa60819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:17.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.543+0000 7fa60d876700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa608106c60 0x7fa60819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:17.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.543+0000 7fa60d876700 1 -- 172.21.15.78:0/2148708969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa60819dba0 msgr2=0x7fa6081977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:17.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.543+0000 7fa60d876700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa60819dba0 0x7fa6081977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:17.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.543+0000 7fa60d876700 1 -- 172.21.15.78:0/2148708969 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6081a44c0 con 0x7fa608197d20 2024-04-25T01:37:17.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.543+0000 7fa60c874700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa60819dba0 0x7fa6081977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:37:17.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.543+0000 7fa60d876700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa608197d20 0x7fa6081981d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa60000bd50 tx=0x7fa60000bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:17.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.543+0000 7fa5fe7fc700 1 -- 172.21.15.78:0/2148708969 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600019070 con 0x7fa608197d20 2024-04-25T01:37:17.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.543+0000 7fa5fe7fc700 1 -- 172.21.15.78:0/2148708969 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa60000cbb0 con 0x7fa608197d20 2024-04-25T01:37:17.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.543+0000 7fa60f2d9700 1 -- 172.21.15.78:0/2148708969 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6081a4750 con 0x7fa608197d20 2024-04-25T01:37:17.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.543+0000 7fa5fe7fc700 1 -- 172.21.15.78:0/2148708969 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600012620 con 0x7fa608197d20 2024-04-25T01:37:17.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.543+0000 7fa60f2d9700 1 -- 172.21.15.78:0/2148708969 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6081a4c90 con 0x7fa608197d20 2024-04-25T01:37:17.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.544+0000 7fa5fe7fc700 1 -- 172.21.15.78:0/2148708969 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa60001c070 con 0x7fa608197d20 2024-04-25T01:37:17.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.544+0000 7fa5fe7fc700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa5f4039580 0x7fa5f403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:17.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.544+0000 7fa5fe7fc700 1 -- 172.21.15.78:0/2148708969 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa60004cac0 con 0x7fa608197d20 2024-04-25T01:37:17.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.544+0000 7fa60f2d9700 1 -- 172.21.15.78:0/2148708969 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa608101160 con 0x7fa608197d20 2024-04-25T01:37:17.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.544+0000 7fa60d075700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa5f4039580 0x7fa5f403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:17.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.545+0000 7fa60d075700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa5f4039580 0x7fa5f403ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa604009750 tx=0x7fa60400b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:17.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.549+0000 7fa5fe7fc700 1 -- 172.21.15.78:0/2148708969 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa6000137a0 con 0x7fa608197d20 2024-04-25T01:37:17.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: mon.b calling monitor election 2024-04-25T01:37:17.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-25T01:37:17.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: Updating smithi152:/etc/ceph/ceph.conf 2024-04-25T01:37:17.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-25T01:37:17.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: mon.b calling monitor election 2024-04-25T01:37:17.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: mon.c calling monitor election 2024-04-25T01:37:17.638 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: mon.a calling monitor election 2024-04-25T01:37:17.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T01:37:17.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: monmap epoch 3 2024-04-25T01:37:17.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: fsid df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:37:17.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: last_changed 2024-04-25T01:37:09.547590+0000 2024-04-25T01:37:17.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: created 2024-04-25T01:35:19.558079+0000 2024-04-25T01:37:17.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: min_mon_release 19 (squid) 2024-04-25T01:37:17.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: election_strategy: 1 2024-04-25T01:37:17.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T01:37:17.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-25T01:37:17.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: 2: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-04-25T01:37:17.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: fsmap 2024-04-25T01:37:17.639 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:37:17.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: mgrmap e13: x(active, since 59s) 2024-04-25T01:37:17.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: overall HEALTH_OK 2024-04-25T01:37:17.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: Updating smithi196:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:37:17.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: Updating smithi152:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:37:17.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: Updating smithi078:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:37:17.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:17.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.640 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:37:17.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:37:17.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.753+0000 7fa60f2d9700 1 -- 172.21.15.78:0/2148708969 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa6081a4f50 con 0x7fa608197d20 2024-04-25T01:37:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.754+0000 7fa5fe7fc700 1 -- 172.21.15.78:0/2148708969 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fa600013a00 con 0x7fa608197d20 2024-04-25T01:37:17.730 INFO:teuthology.orchestra.run.smithi078.stdout:# minimal ceph.conf for df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:37:17.730 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2024-04-25T01:37:17.730 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:37:17.730 INFO:teuthology.orchestra.run.smithi078.stdout: mon_host = [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-04-25T01:37:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.756+0000 7fa60f2d9700 1 -- 172.21.15.78:0/2148708969 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa5f4039580 msgr2=0x7fa5f403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.756+0000 7fa60f2d9700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa5f4039580 0x7fa5f403ba40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa604009750 tx=0x7fa60400b540 comp rx=0 tx=0).stop 2024-04-25T01:37:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.756+0000 7fa60f2d9700 1 -- 172.21.15.78:0/2148708969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa608197d20 msgr2=0x7fa6081981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.756+0000 7fa60f2d9700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa608197d20 0x7fa6081981d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa60000bd50 tx=0x7fa60000bd80 comp rx=0 tx=0).stop 2024-04-25T01:37:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.756+0000 7fa60f2d9700 1 -- 172.21.15.78:0/2148708969 shutdown_connections 2024-04-25T01:37:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.756+0000 7fa60f2d9700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa5f4039580 0x7fa5f403ba40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.756+0000 7fa60f2d9700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa608197d20 0x7fa6081981d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.756+0000 7fa60f2d9700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa608106c60 0x7fa60819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:17.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.756+0000 7fa60f2d9700 1 --2- 172.21.15.78:0/2148708969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa60819dba0 0x7fa6081977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.756+0000 7fa60f2d9700 1 -- 172.21.15.78:0/2148708969 >> 172.21.15.78:0/2148708969 conn(0x7fa6080fb9a0 msgr2=0x7fa608103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.756+0000 7fa60f2d9700 1 -- 172.21.15.78:0/2148708969 shutdown_connections 2024-04-25T01:37:17.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:17.756+0000 7fa60f2d9700 1 -- 172.21.15.78:0/2148708969 wait complete. 2024-04-25T01:37:17.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: mon.b calling monitor election 2024-04-25T01:37:17.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-25T01:37:17.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: Updating smithi152:/etc/ceph/ceph.conf 2024-04-25T01:37:17.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-25T01:37:17.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: mon.b calling monitor election 2024-04-25T01:37:17.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: mon.c calling monitor election 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: mon.a calling monitor election 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: monmap epoch 3 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: fsid df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: last_changed 2024-04-25T01:37:09.547590+0000 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: created 2024-04-25T01:35:19.558079+0000 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: min_mon_release 19 (squid) 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: election_strategy: 1 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: 2: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: fsmap 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: mgrmap e13: x(active, since 59s) 2024-04-25T01:37:17.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: overall HEALTH_OK 2024-04-25T01:37:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: Updating smithi196:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:37:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: Updating smithi152:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:37:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: Updating smithi078:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:37:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:37:17.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:37:17.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:17.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: mon.b calling monitor election 2024-04-25T01:37:17.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: Updating smithi078:/etc/ceph/ceph.conf 2024-04-25T01:37:17.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: Updating smithi152:/etc/ceph/ceph.conf 2024-04-25T01:37:17.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-25T01:37:17.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: mon.b calling monitor election 2024-04-25T01:37:17.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: mon.c calling monitor election 2024-04-25T01:37:17.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: mon.a calling monitor election 2024-04-25T01:37:17.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T01:37:17.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: monmap epoch 3 2024-04-25T01:37:17.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: fsid df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:37:17.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: last_changed 2024-04-25T01:37:09.547590+0000 2024-04-25T01:37:17.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: created 2024-04-25T01:35:19.558079+0000 2024-04-25T01:37:17.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: min_mon_release 19 (squid) 2024-04-25T01:37:17.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: election_strategy: 1 2024-04-25T01:37:17.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-04-25T01:37:17.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-25T01:37:17.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: 2: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-04-25T01:37:17.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: fsmap 2024-04-25T01:37:17.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:37:17.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: mgrmap e13: x(active, since 59s) 2024-04-25T01:37:17.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: overall HEALTH_OK 2024-04-25T01:37:17.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: Updating smithi196:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:37:17.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: Updating smithi152:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:37:17.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: Updating smithi078:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:37:17.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:17.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:17.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:37:17.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:37:17.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:18.342 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T01:37:18.342 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:37:18.342 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T01:37:18.368 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:37:18.368 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:37:18.433 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T01:37:18.433 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T01:37:18.458 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:18 smithi078 ceph-mon[29469]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T01:37:18.458 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:18 smithi078 ceph-mon[29469]: Reconfiguring daemon mon.a on smithi078 2024-04-25T01:37:18.458 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:18.458 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:18 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2148708969' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:18.465 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T01:37:18.465 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:37:18.534 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T01:37:18.535 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T01:37:18.566 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T01:37:18.566 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:37:18.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:18 smithi196 ceph-mon[32978]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T01:37:18.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:18 smithi196 ceph-mon[32978]: Reconfiguring daemon mon.a on smithi078 2024-04-25T01:37:18.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:18.630 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:18 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2148708969' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:18.634 INFO:tasks.cephadm:Adding mgr.x on smithi078 2024-04-25T01:37:19.196 INFO:tasks.cephadm:Adding mgr.y on smithi152 2024-04-25T01:37:19.197 INFO:tasks.cephadm:Adding mgr.z on smithi196 2024-04-25T01:37:19.197 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi078=x;smithi152=y;smithi196=z' 2024-04-25T01:37:19.198 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:18 smithi152 ceph-mon[32465]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T01:37:19.198 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:18 smithi152 ceph-mon[32465]: Reconfiguring daemon mon.a on smithi078 2024-04-25T01:37:19.198 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:37:19.198 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:18 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2148708969' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:19.435 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:37:19.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:19 smithi196 ceph-mon[32978]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:19.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:19.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:19.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:37:19.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:37:19.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:19.727 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:19 smithi152 ceph-mon[32465]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:19.727 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:19.727 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:19.727 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:37:19.727 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:37:19.727 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:19.742 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:19 smithi078 ceph-mon[29469]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:19.742 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:19.742 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:19.742 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:37:19.742 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:37:19.742 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:20.598 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:20 smithi196 ceph-mon[32978]: Reconfiguring mon.b (monmap changed)... 2024-04-25T01:37:20.598 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:20 smithi196 ceph-mon[32978]: Reconfiguring daemon mon.b on smithi152 2024-04-25T01:37:20.681 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:20 smithi152 ceph-mon[32465]: Reconfiguring mon.b (monmap changed)... 2024-04-25T01:37:20.681 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:20 smithi152 ceph-mon[32465]: Reconfiguring daemon mon.b on smithi152 2024-04-25T01:37:20.742 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:20 smithi078 ceph-mon[29469]: Reconfiguring mon.b (monmap changed)... 2024-04-25T01:37:20.742 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:20 smithi078 ceph-mon[29469]: Reconfiguring daemon mon.b on smithi152 2024-04-25T01:37:20.815 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.836+0000 7f051d473700 1 -- 172.21.15.196:0/981976210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0518101dc0 msgr2=0x7f05181021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:20.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.836+0000 7f051d473700 1 --2- 172.21.15.196:0/981976210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0518101dc0 0x7f05181021e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0508009910 tx=0x7f0508009c20 comp rx=0 tx=0).stop 2024-04-25T01:37:20.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.836+0000 7f051d473700 1 -- 172.21.15.196:0/981976210 shutdown_connections 2024-04-25T01:37:20.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.836+0000 7f051d473700 1 --2- 172.21.15.196:0/981976210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0518101dc0 0x7f05181021e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:20.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.836+0000 7f051d473700 1 -- 172.21.15.196:0/981976210 >> 172.21.15.196:0/981976210 conn(0x7f05180fd360 msgr2=0x7f05180ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:20.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.836+0000 7f051d473700 1 -- 172.21.15.196:0/981976210 shutdown_connections 2024-04-25T01:37:20.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.836+0000 7f051d473700 1 -- 172.21.15.196:0/981976210 wait complete. 2024-04-25T01:37:20.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.837+0000 7f051d473700 1 Processor -- start 2024-04-25T01:37:20.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.837+0000 7f051d473700 1 -- start start 2024-04-25T01:37:20.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.837+0000 7f051d473700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0518101dc0 0x7f0518197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:20.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.837+0000 7f051d473700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0518197b50 0x7f051819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:20.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.837+0000 7f051d473700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0518197fd0 0x7f051819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:20.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.837+0000 7f051d473700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0518107e00 con 0x7f0518197b50 2024-04-25T01:37:20.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.837+0000 7f051d473700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0518107c80 con 0x7f0518197fd0 2024-04-25T01:37:20.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.837+0000 7f051d473700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0518107f80 con 0x7f0518101dc0 2024-04-25T01:37:20.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.837+0000 7f0516ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0518101dc0 0x7f0518197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:20.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.837+0000 7f0516ffd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0518101dc0 0x7f0518197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:41558/0 (socket says 172.21.15.196:41558) 2024-04-25T01:37:20.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.837+0000 7f05167fc700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0518197b50 0x7f051819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:20.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.837+0000 7f0516ffd700 1 -- 172.21.15.196:0/2744680912 learned_addr learned my addr 172.21.15.196:0/2744680912 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T01:37:20.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.838+0000 7f05177fe700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0518197fd0 0x7f051819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:20.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.838+0000 7f05167fc700 1 -- 172.21.15.196:0/2744680912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0518101dc0 msgr2=0x7f0518197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:20.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.838+0000 7f05167fc700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0518101dc0 0x7f0518197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:20.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.838+0000 7f05167fc700 1 -- 172.21.15.196:0/2744680912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0518197fd0 msgr2=0x7f051819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:20.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.838+0000 7f05167fc700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0518197fd0 0x7f051819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:20.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.838+0000 7f05167fc700 1 -- 172.21.15.196:0/2744680912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051819fa20 con 0x7f0518197b50 2024-04-25T01:37:20.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.838+0000 7f0516ffd700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0518101dc0 0x7f0518197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:37:20.817 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.838+0000 7f05167fc700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0518197b50 0x7f051819cbc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f050c00b550 tx=0x7f050c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:20.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.838+0000 7f04fffff700 1 -- 172.21.15.196:0/2744680912 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c004230 con 0x7f0518197b50 2024-04-25T01:37:20.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.838+0000 7f04fffff700 1 -- 172.21.15.196:0/2744680912 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050c0043d0 con 0x7f0518197b50 2024-04-25T01:37:20.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.838+0000 7f051d473700 1 -- 172.21.15.196:0/2744680912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051819fd10 con 0x7f0518197b50 2024-04-25T01:37:20.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.838+0000 7f04fffff700 1 -- 172.21.15.196:0/2744680912 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c011b20 con 0x7f0518197b50 2024-04-25T01:37:20.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.838+0000 7f051d473700 1 -- 172.21.15.196:0/2744680912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05181a0270 con 0x7f0518197b50 2024-04-25T01:37:20.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.839+0000 7f04fffff700 1 -- 172.21.15.196:0/2744680912 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f050c011ce0 con 0x7f0518197b50 2024-04-25T01:37:20.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.839+0000 7f04fffff700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0500039580 0x7f050003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:20.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.839+0000 7f04fffff700 1 -- 172.21.15.196:0/2744680912 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f050c04cd20 con 0x7f0518197b50 2024-04-25T01:37:20.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.840+0000 7f051d473700 1 -- 172.21.15.196:0/2744680912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0518063890 con 0x7f0518197b50 2024-04-25T01:37:20.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.840+0000 7f0516ffd700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0500039580 0x7f050003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:20.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.840+0000 7f0516ffd700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0500039580 0x7f050003ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0508005200 tx=0x7f050801a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:20.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:20.845+0000 7f04fffff700 1 -- 172.21.15.196:0/2744680912 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f050c020500 con 0x7f0518197b50 2024-04-25T01:37:21.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:21.060+0000 7f051d473700 1 -- 172.21.15.196:0/2744680912 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi152=y;smithi196=z", "target": ["mon-mgr", ""]}) -- 0x7f0518062700 con 0x7f0500039580 2024-04-25T01:37:21.045 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:21.067+0000 7f04fffff700 1 -- 172.21.15.196:0/2744680912 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0518062700 con 0x7f0500039580 2024-04-25T01:37:21.045 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-04-25T01:37:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:21.069+0000 7f051d473700 1 -- 172.21.15.196:0/2744680912 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0500039580 msgr2=0x7f050003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:21.069+0000 7f051d473700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0500039580 0x7f050003ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0508005200 tx=0x7f050801a040 comp rx=0 tx=0).stop 2024-04-25T01:37:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:21.069+0000 7f051d473700 1 -- 172.21.15.196:0/2744680912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0518197b50 msgr2=0x7f051819cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:21.069+0000 7f051d473700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0518197b50 0x7f051819cbc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f050c00b550 tx=0x7f050c00b860 comp rx=0 tx=0).stop 2024-04-25T01:37:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:21.069+0000 7f051d473700 1 -- 172.21.15.196:0/2744680912 shutdown_connections 2024-04-25T01:37:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:21.069+0000 7f051d473700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0500039580 0x7f050003ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:21.069+0000 7f051d473700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0518101dc0 0x7f0518197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:21.047 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:21.069+0000 7f051d473700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0518197fd0 0x7f051819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:21.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:21.069+0000 7f051d473700 1 --2- 172.21.15.196:0/2744680912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0518197b50 0x7f051819cbc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:21.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:21.069+0000 7f051d473700 1 -- 172.21.15.196:0/2744680912 >> 172.21.15.196:0/2744680912 conn(0x7f05180fd360 msgr2=0x7f05180fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:21.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:21.069+0000 7f051d473700 1 -- 172.21.15.196:0/2744680912 shutdown_connections 2024-04-25T01:37:21.048 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:37:21.069+0000 7f051d473700 1 -- 172.21.15.196:0/2744680912 wait complete. 2024-04-25T01:37:21.640 DEBUG:teuthology.orchestra.run.smithi152:mgr.y> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T01:37:21.643 DEBUG:teuthology.orchestra.run.smithi196:mgr.z> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T01:37:21.645 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T01:37:21.645 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:37:21.645 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T01:37:21.664 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T01:37:21.664 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2024-04-25T01:37:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T01:37:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:37:21.722 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T01:37:21.722 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:37:21.722 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:37:21.722 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-25 01:36:26.728803917 +0000 2024-04-25T01:37:21.722 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-25 01:30:59.612442377 +0000 2024-04-25T01:37:21.722 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-25 01:30:59.612442377 +0000 2024-04-25T01:37:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-25 01:30:59.612442377 +0000 2024-04-25T01:37:21.723 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T01:37:21.790 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-25T01:37:21.790 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-25T01:37:21.790 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.0002398 s, 2.1 MB/s 2024-04-25T01:37:21.792 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T01:37:21.830 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:21 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:21.830 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:21 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:21.830 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:21 smithi196 ceph-mon[32978]: Reconfiguring mon.c (monmap changed)... 2024-04-25T01:37:21.830 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:21 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:37:21.830 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:21 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:37:21.830 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:21 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:21.830 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:21 smithi196 ceph-mon[32978]: Reconfiguring daemon mon.c on smithi196 2024-04-25T01:37:21.830 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:21 smithi196 ceph-mon[32978]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:21.830 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:21 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:21.850 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2024-04-25T01:37:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T01:37:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:37:21.908 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T01:37:21.908 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:37:21.908 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:37:21.908 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-25 01:36:26.762803429 +0000 2024-04-25T01:37:21.909 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-25 01:30:59.965437108 +0000 2024-04-25T01:37:21.909 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-25 01:30:59.965437108 +0000 2024-04-25T01:37:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-25 01:30:59.965437108 +0000 2024-04-25T01:37:21.909 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T01:37:21.975 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-25T01:37:21.975 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-25T01:37:21.976 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000232967 s, 2.2 MB/s 2024-04-25T01:37:21.977 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T01:37:21.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:21 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:21.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:21 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:21.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:21 smithi078 ceph-mon[29469]: Reconfiguring mon.c (monmap changed)... 2024-04-25T01:37:21.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:21 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:37:21.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:21 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:37:21.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:21 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:21.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:21 smithi078 ceph-mon[29469]: Reconfiguring daemon mon.c on smithi196 2024-04-25T01:37:21.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:21 smithi078 ceph-mon[29469]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:21.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:21 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:22.009 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2024-04-25T01:37:22.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:21 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:22.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:21 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:22.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:21 smithi152 ceph-mon[32465]: Reconfiguring mon.c (monmap changed)... 2024-04-25T01:37:22.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:21 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:37:22.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:21 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:37:22.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:21 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:22.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:21 smithi152 ceph-mon[32465]: Reconfiguring daemon mon.c on smithi196 2024-04-25T01:37:22.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:21 smithi152 ceph-mon[32465]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:22.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:21 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T01:37:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:37:22.066 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T01:37:22.067 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:37:22.067 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:37:22.067 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-25 01:36:26.794802971 +0000 2024-04-25T01:37:22.067 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-25 01:31:00.331431646 +0000 2024-04-25T01:37:22.067 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-25 01:31:00.331431646 +0000 2024-04-25T01:37:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-25 01:31:00.331431646 +0000 2024-04-25T01:37:22.067 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T01:37:22.131 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-25T01:37:22.132 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-25T01:37:22.132 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000203276 s, 2.5 MB/s 2024-04-25T01:37:22.133 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T01:37:22.190 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2024-04-25T01:37:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T01:37:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:37:22.246 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T01:37:22.246 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:37:22.246 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:37:22.246 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-25 01:36:26.830802455 +0000 2024-04-25T01:37:22.246 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-25 01:31:00.669426602 +0000 2024-04-25T01:37:22.246 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-25 01:31:00.669426602 +0000 2024-04-25T01:37:22.246 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-25 01:31:00.669426602 +0000 2024-04-25T01:37:22.247 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T01:37:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-25T01:37:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-25T01:37:22.309 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000238638 s, 2.1 MB/s 2024-04-25T01:37:22.310 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T01:37:22.367 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T01:37:22.368 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T01:37:22.386 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T01:37:22.386 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_1 2024-04-25T01:37:22.444 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T01:37:22.444 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:37:22.444 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T01:37:22.444 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:37:22.444 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:37:22.444 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-04-25 01:37:08.988478170 +0000 2024-04-25T01:37:22.444 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-04-25 01:31:09.009528128 +0000 2024-04-25T01:37:22.444 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-04-25 01:31:09.009528128 +0000 2024-04-25T01:37:22.444 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-04-25 01:31:09.009528128 +0000 2024-04-25T01:37:22.444 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T01:37:22.509 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-25T01:37:22.509 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-25T01:37:22.509 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000288501 s, 1.8 MB/s 2024-04-25T01:37:22.511 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T01:37:22.569 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_2 2024-04-25T01:37:22.625 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T01:37:22.625 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:37:22.625 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T01:37:22.625 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:37:22.626 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:37:22.626 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-04-25 01:37:08.988478170 +0000 2024-04-25T01:37:22.626 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-04-25 01:31:09.339524204 +0000 2024-04-25T01:37:22.626 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-04-25 01:31:09.339524204 +0000 2024-04-25T01:37:22.626 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-04-25 01:31:09.339524204 +0000 2024-04-25T01:37:22.626 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T01:37:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-25T01:37:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-25T01:37:22.691 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.00163556 s, 313 kB/s 2024-04-25T01:37:22.693 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T01:37:22.749 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_3 2024-04-25T01:37:22.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:22 smithi152 ceph-mon[32465]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi152=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:22.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:22 smithi152 ceph-mon[32465]: Saving service mgr spec with placement smithi078=x;smithi152=y;smithi196=z;count:3 2024-04-25T01:37:22.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:22 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:22.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:22 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:22.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:22 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:22.799 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T01:37:22.799 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:37:22.799 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T01:37:22.799 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:37:22.799 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:37:22.799 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-04-25 01:37:08.989478159 +0000 2024-04-25T01:37:22.799 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-04-25 01:31:09.676520197 +0000 2024-04-25T01:37:22.799 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-04-25 01:31:09.676520197 +0000 2024-04-25T01:37:22.800 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-04-25 01:31:09.676520197 +0000 2024-04-25T01:37:22.800 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T01:37:22.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:22 smithi196 ceph-mon[32978]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi152=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:22.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:22 smithi196 ceph-mon[32978]: Saving service mgr spec with placement smithi078=x;smithi152=y;smithi196=z;count:3 2024-04-25T01:37:22.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:22 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:22.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:22 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:22.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:22 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:22.863 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-25T01:37:22.863 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-25T01:37:22.863 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000204488 s, 2.5 MB/s 2024-04-25T01:37:22.864 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T01:37:22.921 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_4 2024-04-25T01:37:22.977 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T01:37:22.977 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:37:22.977 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T01:37:22.977 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:37:22.977 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:37:22.977 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-04-25 01:37:08.989478159 +0000 2024-04-25T01:37:22.978 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-04-25 01:31:10.021516095 +0000 2024-04-25T01:37:22.978 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-04-25 01:31:10.021516095 +0000 2024-04-25T01:37:22.978 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-04-25 01:31:10.021516095 +0000 2024-04-25T01:37:22.978 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T01:37:22.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:22 smithi078 ceph-mon[29469]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi152=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:22.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:22 smithi078 ceph-mon[29469]: Saving service mgr spec with placement smithi078=x;smithi152=y;smithi196=z;count:3 2024-04-25T01:37:22.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:22 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:22.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:22 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:22.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:22 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-25T01:37:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-25T01:37:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000191212 s, 2.7 MB/s 2024-04-25T01:37:23.042 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T01:37:23.097 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T01:37:23.097 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T01:37:23.113 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T01:37:23.113 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-04-25T01:37:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T01:37:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:37:23.168 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T01:37:23.168 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:37:23.168 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:37:23.168 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-25 01:36:58.889493539 +0000 2024-04-25T01:37:23.168 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-25 01:31:26.991801697 +0000 2024-04-25T01:37:23.168 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-25 01:31:26.991801697 +0000 2024-04-25T01:37:23.168 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-25 01:31:26.991801697 +0000 2024-04-25T01:37:23.169 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T01:37:23.231 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-25T01:37:23.231 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-25T01:37:23.231 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000344518 s, 1.5 MB/s 2024-04-25T01:37:23.232 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T01:37:23.288 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-04-25T01:37:23.343 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T01:37:23.343 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:37:23.343 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T01:37:23.343 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:37:23.343 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:37:23.343 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-25 01:36:58.889493539 +0000 2024-04-25T01:37:23.343 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-25 01:31:27.337793869 +0000 2024-04-25T01:37:23.343 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-25 01:31:27.337793869 +0000 2024-04-25T01:37:23.343 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-25 01:31:27.337793869 +0000 2024-04-25T01:37:23.344 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T01:37:23.405 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-25T01:37:23.405 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-25T01:37:23.405 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000190009 s, 2.7 MB/s 2024-04-25T01:37:23.407 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T01:37:23.462 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-04-25T01:37:23.517 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T01:37:23.517 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:37:23.517 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T01:37:23.517 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:37:23.517 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:37:23.517 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-25 01:36:58.890493517 +0000 2024-04-25T01:37:23.517 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-25 01:31:27.688785928 +0000 2024-04-25T01:37:23.518 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-25 01:31:27.688785928 +0000 2024-04-25T01:37:23.518 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-25 01:31:27.688785928 +0000 2024-04-25T01:37:23.518 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T01:37:23.580 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-25T01:37:23.580 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-25T01:37:23.580 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000202287 s, 2.5 MB/s 2024-04-25T01:37:23.582 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T01:37:23.638 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-04-25T01:37:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T01:37:23.692 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:37:23.693 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T01:37:23.693 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:37:23.693 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:37:23.693 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-25 01:36:58.890493517 +0000 2024-04-25T01:37:23.693 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-25 01:31:28.040777965 +0000 2024-04-25T01:37:23.693 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-25 01:31:28.040777965 +0000 2024-04-25T01:37:23.693 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-25 01:31:28.040777965 +0000 2024-04-25T01:37:23.693 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T01:37:23.756 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-25T01:37:23.756 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-25T01:37:23.756 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000256327 s, 2.0 MB/s 2024-04-25T01:37:23.757 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T01:37:23.814 INFO:tasks.cephadm:Deploying osd.0 on smithi078 with /dev/vg_nvme/lv_4... 2024-04-25T01:37:23.814 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T01:37:23.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:23 smithi196 ceph-mon[32978]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:23.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:23 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:23.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:23 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:23.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:23 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:23.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:23 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:23.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:23 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T01:37:23.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:23 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:37:23.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:23 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:24.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:23 smithi152 ceph-mon[32465]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:24.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:23 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:24.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:23 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:24.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:23 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:24.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:23 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:24.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:23 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T01:37:24.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:23 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:37:24.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:23 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:24.043 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:37:24.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:23 smithi078 ceph-mon[29469]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:24.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:23 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:24.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:23 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:24.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:23 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:24.163 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:23 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:24.163 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:23 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T01:37:24.163 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:23 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:37:24.163 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:23 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:24.792 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:24 smithi078 ceph-mon[29469]: Deploying daemon mgr.z on smithi196 2024-04-25T01:37:25.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:24 smithi152 ceph-mon[32465]: Deploying daemon mgr.z on smithi196 2024-04-25T01:37:25.062 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:24 smithi196 ceph-mon[32978]: Deploying daemon mgr.z on smithi196 2024-04-25T01:37:25.352 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:25 smithi196 systemd[1]: Starting Ceph mgr.z for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:37:25.745 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:25 smithi196 podman[34814]: 2024-04-25 01:37:25.402410744 +0000 UTC m=+0.017550366 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:37:25.746 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:25 smithi196 podman[34814]: 2024-04-25 01:37:25.533453627 +0000 UTC m=+0.148593272 container create ab555bd66c8c1ac920c1ab0bb174999b9cb5d807ab517f1c6dc0eb059f5b16a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T01:37:25.746 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:25 smithi196 podman[34814]: 2024-04-25 01:37:25.769244568 +0000 UTC m=+0.384384170 container init ab555bd66c8c1ac920c1ab0bb174999b9cb5d807ab517f1c6dc0eb059f5b16a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:37:25.936 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T01:37:25.937 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T01:37:25.937 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-25T01:37:25.937 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-25T01:37:25.937 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115267 s, 910 MB/s 2024-04-25T01:37:25.937 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-25T01:37:26.034 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:25 smithi152 ceph-mon[32465]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:26.082 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:25 smithi196 ceph-mon[32978]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:26.082 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:25 smithi196 podman[34814]: 2024-04-25 01:37:25.772033973 +0000 UTC m=+0.387173582 container start ab555bd66c8c1ac920c1ab0bb174999b9cb5d807ab517f1c6dc0eb059f5b16a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-25T01:37:26.083 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:25 smithi196 ceph-mgr[34833]: -- 172.21.15.196:0/3619768563 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556ec2a445a0 con 0x556ec2a18c00 2024-04-25T01:37:26.083 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:25 smithi196 bash[34814]: ab555bd66c8c1ac920c1ab0bb174999b9cb5d807ab517f1c6dc0eb059f5b16a0 2024-04-25T01:37:26.083 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:25 smithi196 systemd[1]: Started Ceph mgr.z for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:37:26.083 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:25 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:25.917+0000 7ff428fde200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:37:26.083 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:26.104+0000 7ff428fde200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:37:26.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:25 smithi078 ceph-mon[29469]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:26.352 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:26.204+0000 7ff428fde200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:37:26.483 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_4 2024-04-25T01:37:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:37:26.852 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:26.506+0000 7ff428fde200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:37:26.852 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:26.589+0000 7ff428fde200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:37:27.151 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:26 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:27.151 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:26 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:27.151 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:26 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:27.152 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:26 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:27.152 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:26 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T01:37:27.152 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:26 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:37:27.152 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:26 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:27.152 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:26 smithi078 ceph-mon[29469]: Deploying daemon mgr.y on smithi152 2024-04-25T01:37:27.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:26 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:27.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:26 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:27.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:26 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:27.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:26 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:27.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:26 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T01:37:27.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:26 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:37:27.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:26 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:27.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:26 smithi152 ceph-mon[32465]: Deploying daemon mgr.y on smithi152 2024-04-25T01:37:27.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:26 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:27.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:26 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:27.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:26 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:27.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:26 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:27.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:26 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T01:37:27.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:26 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:37:27.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:26 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:27.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:26 smithi196 ceph-mon[32978]: Deploying daemon mgr.y on smithi152 2024-04-25T01:37:27.955 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:27 smithi152 systemd[1]: Starting Ceph mgr.y for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:37:28.083 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:27 smithi196 ceph-mon[32978]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:28.083 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:27 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:27.761+0000 7ff428fde200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:37:28.083 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:27 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:27.927+0000 7ff428fde200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:37:28.083 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:28.104+0000 7ff428fde200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:37:28.148 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:27 smithi078 ceph-mon[29469]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:28.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.208+0000 7f0e70628700 1 -- 172.21.15.78:0/3526659336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e68103490 msgr2=0x7f0e681038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:28.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.208+0000 7f0e70628700 1 --2- 172.21.15.78:0/3526659336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e68103490 0x7f0e681038f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0e580099c0 tx=0x7f0e58009cd0 comp rx=0 tx=0).stop 2024-04-25T01:37:28.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.208+0000 7f0e70628700 1 -- 172.21.15.78:0/3526659336 shutdown_connections 2024-04-25T01:37:28.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.208+0000 7f0e70628700 1 --2- 172.21.15.78:0/3526659336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e68102290 0x7f0e681026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:28.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.208+0000 7f0e70628700 1 --2- 172.21.15.78:0/3526659336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e68103e30 0x7f0e6810a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:28.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.208+0000 7f0e70628700 1 --2- 172.21.15.78:0/3526659336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e68103490 0x7f0e681038f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:28.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.208+0000 7f0e70628700 1 -- 172.21.15.78:0/3526659336 >> 172.21.15.78:0/3526659336 conn(0x7f0e680fd830 msgr2=0x7f0e680ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:28.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.208+0000 7f0e70628700 1 -- 172.21.15.78:0/3526659336 shutdown_connections 2024-04-25T01:37:28.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.208+0000 7f0e70628700 1 -- 172.21.15.78:0/3526659336 wait complete. 2024-04-25T01:37:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e70628700 1 Processor -- start 2024-04-25T01:37:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e70628700 1 -- start start 2024-04-25T01:37:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e70628700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e68102290 0x7f0e681984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e70628700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e68103490 0x7f0e68198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e70628700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e68103e30 0x7f0e6819faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e70628700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6810de70 con 0x7f0e68103e30 2024-04-25T01:37:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e70628700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6810dcf0 con 0x7f0e68103490 2024-04-25T01:37:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e70628700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6810dff0 con 0x7f0e68102290 2024-04-25T01:37:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e6ebc5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e68103e30 0x7f0e6819faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e6ebc5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e68103e30 0x7f0e6819faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46266/0 (socket says 172.21.15.78:46266) 2024-04-25T01:37:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e6e3c4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e68102290 0x7f0e681984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e6ebc5700 1 -- 172.21.15.78:0/1681776224 learned_addr learned my addr 172.21.15.78:0/1681776224 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:37:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e6dbc3700 1 --2- 172.21.15.78:0/1681776224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e68103490 0x7f0e68198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e6ebc5700 1 -- 172.21.15.78:0/1681776224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e68102290 msgr2=0x7f0e681984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e6ebc5700 1 --2- 172.21.15.78:0/1681776224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e68102290 0x7f0e681984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e6ebc5700 1 -- 172.21.15.78:0/1681776224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e68103490 msgr2=0x7f0e68198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e6ebc5700 1 --2- 172.21.15.78:0/1681776224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e68103490 0x7f0e68198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e6ebc5700 1 -- 172.21.15.78:0/1681776224 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e681a0140 con 0x7f0e68103e30 2024-04-25T01:37:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.209+0000 7f0e6e3c4700 1 --2- 172.21.15.78:0/1681776224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e68102290 0x7f0e681984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:37:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.210+0000 7f0e6ebc5700 1 --2- 172.21.15.78:0/1681776224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e68103e30 0x7f0e6819faa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400d9b0 tx=0x7f0e6400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.210+0000 7f0e5f7fe700 1 -- 172.21.15.78:0/1681776224 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e64016070 con 0x7f0e68103e30 2024-04-25T01:37:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.210+0000 7f0e5f7fe700 1 -- 172.21.15.78:0/1681776224 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e64004d10 con 0x7f0e68103e30 2024-04-25T01:37:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.210+0000 7f0e70628700 1 -- 172.21.15.78:0/1681776224 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e681a0430 con 0x7f0e68103e30 2024-04-25T01:37:28.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.210+0000 7f0e5f7fe700 1 -- 172.21.15.78:0/1681776224 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e64005560 con 0x7f0e68103e30 2024-04-25T01:37:28.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.210+0000 7f0e70628700 1 -- 172.21.15.78:0/1681776224 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e681a0970 con 0x7f0e68103e30 2024-04-25T01:37:28.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.210+0000 7f0e5f7fe700 1 -- 172.21.15.78:0/1681776224 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0e6400b750 con 0x7f0e68103e30 2024-04-25T01:37:28.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.211+0000 7f0e5f7fe700 1 --2- 172.21.15.78:0/1681776224 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0e54039330 0x7f0e5403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:28.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.211+0000 7f0e5f7fe700 1 -- 172.21.15.78:0/1681776224 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0e6404c0d0 con 0x7f0e68103e30 2024-04-25T01:37:28.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.211+0000 7f0e6e3c4700 1 --2- 172.21.15.78:0/1681776224 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0e54039330 0x7f0e5403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:28.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.211+0000 7f0e70628700 1 -- 172.21.15.78:0/1681776224 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e68068f20 con 0x7f0e68103e30 2024-04-25T01:37:28.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.211+0000 7f0e6e3c4700 1 --2- 172.21.15.78:0/1681776224 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0e54039330 0x7f0e5403b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0e681032f0 tx=0x7f0e6000b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:28.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.216+0000 7f0e5f7fe700 1 -- 172.21.15.78:0/1681776224 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0e6401f020 con 0x7f0e68103e30 2024-04-25T01:37:28.219 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:27 smithi152 ceph-mon[32465]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:28.219 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:28 smithi152 podman[33836]: 2024-04-25 01:37:27.919877362 +0000 UTC m=+0.021913921 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:37:28.219 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:28 smithi152 podman[33836]: 2024-04-25 01:37:28.029093126 +0000 UTC m=+0.131129670 container create 270917cafea24cb7a6a59480a80fb218068bc49f60f80b38e2149ecb739b81fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T01:37:28.219 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:28 smithi152 podman[33836]: 2024-04-25 01:37:28.214670351 +0000 UTC m=+0.316706900 container init 270917cafea24cb7a6a59480a80fb218068bc49f60f80b38e2149ecb739b81fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T01:37:28.219 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:28 smithi152 podman[33836]: 2024-04-25 01:37:28.217448863 +0000 UTC m=+0.319485411 container start 270917cafea24cb7a6a59480a80fb218068bc49f60f80b38e2149ecb739b81fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:37:28.220 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:28 smithi152 ceph-mgr[33857]: -- 172.21.15.152:0/1807858009 <== mon.2 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c5ecee85a0 con 0x55c5ecebcc00 2024-04-25T01:37:28.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:28.426+0000 7f0e70628700 1 -- 172.21.15.78:0/1681776224 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0e68062670 con 0x7f0e54039330 2024-04-25T01:37:28.534 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:28 smithi152 bash[33836]: 270917cafea24cb7a6a59480a80fb218068bc49f60f80b38e2149ecb739b81fb 2024-04-25T01:37:28.534 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:28 smithi152 systemd[1]: Started Ceph mgr.y for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:37:28.534 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:28 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:28.372+0000 7ff31a4a8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:37:28.690 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:28.403+0000 7ff428fde200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:37:28.691 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:28.499+0000 7ff428fde200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:37:28.691 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:28.712+0000 7ff428fde200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:37:28.995 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:28 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:28.572+0000 7ff31a4a8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:37:28.995 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:28 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:28.686+0000 7ff31a4a8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:37:28.995 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:29 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:29.018+0000 7ff31a4a8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:37:29.102 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:28.912+0000 7ff428fde200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:37:29.284 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:29 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:29.115+0000 7ff31a4a8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:37:29.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:29.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:29.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:29.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:29.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:29 smithi196 ceph-mon[32978]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:29.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:37:29.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:37:29.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:29.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:29.528 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:29 smithi196 ceph-mon[32978]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:29.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:29.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:29.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:29.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:29.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:29 smithi078 ceph-mon[29469]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:29.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:37:29.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:37:29.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:29.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:29.614 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:29 smithi078 ceph-mon[29469]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:29.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:29.784 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:29.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:29.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:29.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:29 smithi152 ceph-mon[32465]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:29.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:37:29.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:37:29.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:29.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:29.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:29 smithi152 ceph-mon[32465]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:29.852 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:29 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:29.562+0000 7ff428fde200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T01:37:30.287 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:30.012+0000 7ff428fde200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:37:30.287 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:30.109+0000 7ff428fde200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:37:30.288 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:30.309+0000 7ff428fde200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T01:37:30.697 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:30 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:30.381+0000 7ff31a4a8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:37:30.697 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:30 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:30.549+0000 7ff31a4a8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:37:30.697 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:30 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:30.720+0000 7ff31a4a8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:37:30.988 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:30 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:30.989 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:30 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:30.989 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:30 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:30.989 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:30 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:30.989 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:30 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:30.989 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:30 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:30.989 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:30 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:30.989 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:30 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:30.989 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:30 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:37:30.989 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:30 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:31.102 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:30.836+0000 7ff428fde200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:37:31.102 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:30.928+0000 7ff428fde200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:37:31.102 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:31.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:31.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:31.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:31.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:31.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:31.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:31.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:31.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:37:31.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:30 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:31.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:30 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:31.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:30 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:31.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:30 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:31.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:30 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:31.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:30 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:31.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:30 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:31.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:30 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:31.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:30 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:31.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:30 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:37:31.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:30 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:31.284 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:31 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:31.011+0000 7ff31a4a8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:37:31.284 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:31 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:31.103+0000 7ff31a4a8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:37:31.487 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:31 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:31.167+0000 7ff428fde200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:37:31.487 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:31 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:31.299+0000 7ff428fde200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:37:31.487 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:31 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:31.395+0000 7ff428fde200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:37:31.487 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:31 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:31.508+0000 7ff428fde200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:37:31.784 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:31 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:31.308+0000 7ff31a4a8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:37:31.784 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:31 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:31.499+0000 7ff31a4a8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:37:31.852 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:31 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:31.599+0000 7ff428fde200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:37:32.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:31 smithi078 ceph-mon[29469]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T01:37:32.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:31 smithi078 ceph-mon[29469]: Reconfiguring daemon mgr.x on smithi078 2024-04-25T01:37:32.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:31 smithi078 ceph-mon[29469]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:32.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:31 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2324402565' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bab2e70b-bf93-48c8-944f-00e2d8424353"}]: dispatch 2024-04-25T01:37:32.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:31 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2324402565' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bab2e70b-bf93-48c8-944f-00e2d8424353"}]': finished 2024-04-25T01:37:32.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:31 smithi078 ceph-mon[29469]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T01:37:32.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:31 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:32.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:31 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1194485862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:37:32.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:31 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:32.284 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:32 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:32.105+0000 7ff31a4a8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T01:37:32.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:31 smithi152 ceph-mon[32465]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T01:37:32.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:31 smithi152 ceph-mon[32465]: Reconfiguring daemon mgr.x on smithi078 2024-04-25T01:37:32.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:31 smithi152 ceph-mon[32465]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:32.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:31 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2324402565' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bab2e70b-bf93-48c8-944f-00e2d8424353"}]: dispatch 2024-04-25T01:37:32.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:31 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2324402565' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bab2e70b-bf93-48c8-944f-00e2d8424353"}]': finished 2024-04-25T01:37:32.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:31 smithi152 ceph-mon[32465]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T01:37:32.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:31 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:32.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:31 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1194485862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:37:32.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:31 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:32.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:31 smithi196 ceph-mon[32978]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T01:37:32.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:31 smithi196 ceph-mon[32978]: Reconfiguring daemon mgr.x on smithi078 2024-04-25T01:37:32.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:31 smithi196 ceph-mon[32978]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:32.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:31 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2324402565' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bab2e70b-bf93-48c8-944f-00e2d8424353"}]: dispatch 2024-04-25T01:37:32.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:31 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2324402565' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bab2e70b-bf93-48c8-944f-00e2d8424353"}]': finished 2024-04-25T01:37:32.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:31 smithi196 ceph-mon[32978]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T01:37:32.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:31 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:32.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:31 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1194485862' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:37:32.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:31 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:32.784 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:32 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:32.528+0000 7ff31a4a8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:37:32.784 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:32 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:32.624+0000 7ff31a4a8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:37:32.852 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:32 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:32.404+0000 7ff428fde200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:37:32.852 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:37:32 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z[34829]: 2024-04-25T01:37:32.616+0000 7ff428fde200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:37:32.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:32.894+0000 7f0e5f7fe700 1 -- 172.21.15.78:0/1681776224 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f0e64003a00 con 0x7f0e68103e30 2024-04-25T01:37:33.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:32 smithi078 ceph-mon[29469]: Standby manager daemon z started 2024-04-25T01:37:33.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:32 smithi078 ceph-mon[29469]: from='mgr.? 172.21.15.196:0/488299728' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T01:37:33.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:32 smithi078 ceph-mon[29469]: from='mgr.? 172.21.15.196:0/488299728' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:37:33.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:32 smithi078 ceph-mon[29469]: from='mgr.? 172.21.15.196:0/488299728' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T01:37:33.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:32 smithi078 ceph-mon[29469]: from='mgr.? 172.21.15.196:0/488299728' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:37:33.284 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:32 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:32.824+0000 7ff31a4a8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T01:37:33.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:32 smithi152 ceph-mon[32465]: Standby manager daemon z started 2024-04-25T01:37:33.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:32 smithi152 ceph-mon[32465]: from='mgr.? 172.21.15.196:0/488299728' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T01:37:33.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:32 smithi152 ceph-mon[32465]: from='mgr.? 172.21.15.196:0/488299728' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:37:33.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:32 smithi152 ceph-mon[32465]: from='mgr.? 172.21.15.196:0/488299728' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T01:37:33.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:32 smithi152 ceph-mon[32465]: from='mgr.? 172.21.15.196:0/488299728' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:37:33.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:32 smithi196 ceph-mon[32978]: Standby manager daemon z started 2024-04-25T01:37:33.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:32 smithi196 ceph-mon[32978]: from='mgr.? 172.21.15.196:0/488299728' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T01:37:33.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:32 smithi196 ceph-mon[32978]: from='mgr.? 172.21.15.196:0/488299728' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:37:33.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:32 smithi196 ceph-mon[32978]: from='mgr.? 172.21.15.196:0/488299728' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T01:37:33.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:32 smithi196 ceph-mon[32978]: from='mgr.? 172.21.15.196:0/488299728' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:37:33.661 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:33 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:33.351+0000 7ff31a4a8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:37:33.661 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:33 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:33.440+0000 7ff31a4a8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:37:34.006 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:33 smithi152 ceph-mon[32465]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:34.006 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:33 smithi152 ceph-mon[32465]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-25T01:37:34.006 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:33 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T01:37:34.006 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:33 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:33.683+0000 7ff31a4a8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:37:34.007 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:33 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:33.816+0000 7ff31a4a8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:37:34.007 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:33 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:33.912+0000 7ff31a4a8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:37:34.007 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:34 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:34.028+0000 7ff31a4a8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:37:34.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:33 smithi078 ceph-mon[29469]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:34.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:33 smithi078 ceph-mon[29469]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-25T01:37:34.243 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:33 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T01:37:34.284 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:34 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:34.120+0000 7ff31a4a8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:37:34.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:33 smithi196 ceph-mon[32978]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:34.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:33 smithi196 ceph-mon[32978]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-25T01:37:34.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:33 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T01:37:35.284 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:34 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:34.934+0000 7ff31a4a8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:37:35.285 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:37:35 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y[33852]: 2024-04-25T01:37:35.149+0000 7ff31a4a8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:37:35.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:35.902+0000 7f0e5f7fe700 1 -- 172.21.15.78:0/1681776224 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f0e6405e5d0 con 0x7f0e68103e30 2024-04-25T01:37:35.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:35 smithi078 ceph-mon[29469]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:35.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:35 smithi078 ceph-mon[29469]: Standby manager daemon y started 2024-04-25T01:37:35.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:35 smithi078 ceph-mon[29469]: from='mgr.? 172.21.15.152:0/2279892628' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T01:37:35.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:35 smithi078 ceph-mon[29469]: from='mgr.? 172.21.15.152:0/2279892628' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:37:35.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:35 smithi078 ceph-mon[29469]: from='mgr.? 172.21.15.152:0/2279892628' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T01:37:35.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:35 smithi078 ceph-mon[29469]: from='mgr.? 172.21.15.152:0/2279892628' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:37:36.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:35 smithi152 ceph-mon[32465]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:36.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:35 smithi152 ceph-mon[32465]: Standby manager daemon y started 2024-04-25T01:37:36.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:35 smithi152 ceph-mon[32465]: from='mgr.? 172.21.15.152:0/2279892628' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T01:37:36.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:35 smithi152 ceph-mon[32465]: from='mgr.? 172.21.15.152:0/2279892628' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:37:36.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:35 smithi152 ceph-mon[32465]: from='mgr.? 172.21.15.152:0/2279892628' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T01:37:36.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:35 smithi152 ceph-mon[32465]: from='mgr.? 172.21.15.152:0/2279892628' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:37:36.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:35 smithi196 ceph-mon[32978]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:36.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:35 smithi196 ceph-mon[32978]: Standby manager daemon y started 2024-04-25T01:37:36.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:35 smithi196 ceph-mon[32978]: from='mgr.? 172.21.15.152:0/2279892628' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T01:37:36.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:35 smithi196 ceph-mon[32978]: from='mgr.? 172.21.15.152:0/2279892628' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:37:36.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:35 smithi196 ceph-mon[32978]: from='mgr.? 172.21.15.152:0/2279892628' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T01:37:36.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:35 smithi196 ceph-mon[32978]: from='mgr.? 172.21.15.152:0/2279892628' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:37:36.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:36 smithi078 ceph-mon[29469]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T01:37:36.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:36 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T01:37:36.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:36 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:36.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:36 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:36.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:36 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:36.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:36 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:36.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:36 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:36.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:36 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:37.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:36 smithi152 ceph-mon[32465]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T01:37:37.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:36 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T01:37:37.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:36 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:37.286 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:36 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:37.286 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:36 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:37.286 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:36 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:37.286 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:36 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:37.286 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:36 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:37.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:36 smithi196 ceph-mon[32978]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T01:37:37.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:36 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T01:37:37.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:36 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:37.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:36 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:37.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:36 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:37.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:36 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:37.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:36 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:37.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:36 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:37.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:37 smithi078 ceph-mon[29469]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:38.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:37 smithi152 ceph-mon[32465]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:38.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:37 smithi196 ceph-mon[32978]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:38.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:38 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T01:37:38.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:38 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:39.102 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:38 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T01:37:39.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:38 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:39.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:38 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T01:37:39.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:38 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:40.146 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:39 smithi078 ceph-mon[29469]: Deploying daemon osd.0 on smithi078 2024-04-25T01:37:40.146 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:39 smithi078 ceph-mon[29469]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:40.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:39 smithi152 ceph-mon[32465]: Deploying daemon osd.0 on smithi078 2024-04-25T01:37:40.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:39 smithi152 ceph-mon[32465]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:40.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:39 smithi196 ceph-mon[32978]: Deploying daemon osd.0 on smithi078 2024-04-25T01:37:40.352 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:39 smithi196 ceph-mon[32978]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:41.993 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:41 smithi078 ceph-mon[29469]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:42.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:41 smithi152 ceph-mon[32465]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:42.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:41 smithi196 ceph-mon[32978]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:44.195 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:43 smithi078 ceph-mon[29469]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:44.195 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:43 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:44.195 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:43 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:44.196 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:43 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:44.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:43 smithi152 ceph-mon[32465]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:44.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:43 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:44.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:43 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:44.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:43 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:44.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:43 smithi196 ceph-mon[32978]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:44.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:43 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:44.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:43 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:44.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:43 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:46.205 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:45 smithi078 ceph-mon[29469]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:46.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:45 smithi152 ceph-mon[32465]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:46.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:45 smithi196 ceph-mon[32978]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:47.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:47.483+0000 7f0e5f7fe700 1 -- 172.21.15.78:0/1681776224 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0e68062670 con 0x7f0e54039330 2024-04-25T01:37:47.462 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 0 on host 'smithi078' 2024-04-25T01:37:47.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:47.485+0000 7f0e70628700 1 -- 172.21.15.78:0/1681776224 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0e54039330 msgr2=0x7f0e5403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:47.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:47.485+0000 7f0e70628700 1 --2- 172.21.15.78:0/1681776224 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0e54039330 0x7f0e5403b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0e681032f0 tx=0x7f0e6000b040 comp rx=0 tx=0).stop 2024-04-25T01:37:47.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:47.485+0000 7f0e70628700 1 -- 172.21.15.78:0/1681776224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e68103e30 msgr2=0x7f0e6819faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:47.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:47.485+0000 7f0e70628700 1 --2- 172.21.15.78:0/1681776224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e68103e30 0x7f0e6819faa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400d9b0 tx=0x7f0e6400dcc0 comp rx=0 tx=0).stop 2024-04-25T01:37:47.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:47.485+0000 7f0e70628700 1 -- 172.21.15.78:0/1681776224 shutdown_connections 2024-04-25T01:37:47.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:47.485+0000 7f0e70628700 1 --2- 172.21.15.78:0/1681776224 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0e54039330 0x7f0e5403b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:47.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:47.485+0000 7f0e70628700 1 --2- 172.21.15.78:0/1681776224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e68102290 0x7f0e681984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:47.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:47.485+0000 7f0e70628700 1 --2- 172.21.15.78:0/1681776224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e68103e30 0x7f0e6819faa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:47.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:47.485+0000 7f0e70628700 1 --2- 172.21.15.78:0/1681776224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e68103490 0x7f0e68198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:47.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:47.485+0000 7f0e70628700 1 -- 172.21.15.78:0/1681776224 >> 172.21.15.78:0/1681776224 conn(0x7f0e680fd830 msgr2=0x7f0e681067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:47.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:47.485+0000 7f0e70628700 1 -- 172.21.15.78:0/1681776224 shutdown_connections 2024-04-25T01:37:47.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:37:47.485+0000 7f0e70628700 1 -- 172.21.15.78:0/1681776224 wait complete. 2024-04-25T01:37:47.666 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:47.666 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:47.666 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:47.666 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:47.666 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:47.666 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:47 smithi078 ceph-mon[29469]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:47.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:47 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:47.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:47 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:47.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:47 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:47.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:47 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:47.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:47 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:47.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:47 smithi152 ceph-mon[32465]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:47.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:47 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:47.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:47 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:47.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:47 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:47.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:47 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:47.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:47 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:47.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:47 smithi196 ceph-mon[32978]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:48.111 DEBUG:teuthology.orchestra.run.smithi078:osd.0> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T01:37:48.114 INFO:tasks.cephadm:Deploying osd.1 on smithi152 with /dev/vg_nvme/lv_4... 2024-04-25T01:37:48.114 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T01:37:48.360 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:37:48.699 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:48 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:48.699 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:48 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:48.699 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:48 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:48.699 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:48 smithi152 ceph-mon[32465]: from='osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T01:37:48.743 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:48 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:48.743 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:48 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:48.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:48 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:48.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:48 smithi078 ceph-mon[29469]: from='osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T01:37:48.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:48 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:48.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:48 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:48.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:48 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:48.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:48 smithi196 ceph-mon[32978]: from='osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T01:37:49.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:49 smithi078 ceph-mon[29469]: from='osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T01:37:49.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:49 smithi078 ceph-mon[29469]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T01:37:49.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:49 smithi078 ceph-mon[29469]: from='osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T01:37:49.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:49.550 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:49 smithi078 ceph-mon[29469]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:49.550 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:37:49 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-0[41942]: 2024-04-25T01:37:49.483+0000 7f9de7638700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T01:37:49.551 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:37:49 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-0[41942]: 2024-04-25T01:37:49.485+0000 7f9de342b700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:37:49.578 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:49 smithi152 ceph-mon[32465]: from='osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T01:37:49.578 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:49 smithi152 ceph-mon[32465]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T01:37:49.578 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:49 smithi152 ceph-mon[32465]: from='osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T01:37:49.578 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:49.578 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:49 smithi152 ceph-mon[32465]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:49.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:49 smithi196 ceph-mon[32978]: from='osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T01:37:49.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:49 smithi196 ceph-mon[32978]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T01:37:49.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:49 smithi196 ceph-mon[32978]: from='osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T01:37:49.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:49.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:49 smithi196 ceph-mon[32978]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:37:50.158 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T01:37:50.158 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T01:37:50.158 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-04-25T01:37:50.158 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-04-25T01:37:50.158 INFO:teuthology.orchestra.run.smithi152.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0133783 s, 784 MB/s 2024-04-25T01:37:50.158 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-04-25T01:37:50.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:50 smithi078 ceph-mon[29469]: from='osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T01:37:50.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:50 smithi078 ceph-mon[29469]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T01:37:50.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:50 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:50.631 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:50 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:50.778 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:50 smithi152 ceph-mon[32465]: from='osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T01:37:50.778 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:50 smithi152 ceph-mon[32465]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T01:37:50.778 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:50 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:50.778 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:50 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:50.805 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi152:vg_nvme/lv_4 2024-04-25T01:37:50.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:50 smithi196 ceph-mon[32978]: from='osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T01:37:50.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:50 smithi196 ceph-mon[32978]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T01:37:50.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:50 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:50.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:50 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:51.042 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:37:51.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:51 smithi078 ceph-mon[29469]: purged_snaps scrub starts 2024-04-25T01:37:51.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:51 smithi078 ceph-mon[29469]: purged_snaps scrub ok 2024-04-25T01:37:51.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:51 smithi078 ceph-mon[29469]: osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653] boot 2024-04-25T01:37:51.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:51 smithi078 ceph-mon[29469]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T01:37:51.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:51.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:51 smithi078 ceph-mon[29469]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:37:51.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:51.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:51.494 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:37:51.495 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:51.495 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:51.495 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:51.495 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:51.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:51 smithi152 ceph-mon[32465]: purged_snaps scrub starts 2024-04-25T01:37:51.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:51 smithi152 ceph-mon[32465]: purged_snaps scrub ok 2024-04-25T01:37:51.785 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:51 smithi152 ceph-mon[32465]: osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653] boot 2024-04-25T01:37:51.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:51 smithi152 ceph-mon[32465]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T01:37:51.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:51.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:51 smithi152 ceph-mon[32465]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:37:51.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:51.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:51.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:37:51.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:51.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:51.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:51.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:51.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:51 smithi196 ceph-mon[32978]: purged_snaps scrub starts 2024-04-25T01:37:51.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:51 smithi196 ceph-mon[32978]: purged_snaps scrub ok 2024-04-25T01:37:51.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:51 smithi196 ceph-mon[32978]: osd.0 [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653] boot 2024-04-25T01:37:51.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:51 smithi196 ceph-mon[32978]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T01:37:51.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:51.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:51 smithi196 ceph-mon[32978]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:37:51.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:51.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:51.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:37:51.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:51.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:51.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:51.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:37:52.535 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:52 smithi152 ceph-mon[32465]: Detected new or changed devices on smithi078 2024-04-25T01:37:52.536 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:52 smithi152 ceph-mon[32465]: Adjusting osd_memory_target on smithi078 to 17088M 2024-04-25T01:37:52.536 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:52 smithi152 ceph-mon[32465]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T01:37:52.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.577+0000 7f03a80e1700 1 -- 172.21.15.152:0/2344665113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0102390 msgr2=0x7f03a01027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:52.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.577+0000 7f03a80e1700 1 --2- 172.21.15.152:0/2344665113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0102390 0x7f03a01027f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f039c0099c0 tx=0x7f039c009cd0 comp rx=0 tx=0).stop 2024-04-25T01:37:52.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.577+0000 7f03a80e1700 1 -- 172.21.15.152:0/2344665113 shutdown_connections 2024-04-25T01:37:52.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.577+0000 7f03a80e1700 1 --2- 172.21.15.152:0/2344665113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03a0101190 0x7f03a01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:52.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.577+0000 7f03a80e1700 1 --2- 172.21.15.152:0/2344665113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03a0102d30 0x7f03a01095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:52.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.577+0000 7f03a80e1700 1 --2- 172.21.15.152:0/2344665113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0102390 0x7f03a01027f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:52.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.577+0000 7f03a80e1700 1 -- 172.21.15.152:0/2344665113 >> 172.21.15.152:0/2344665113 conn(0x7f03a00fc730 msgr2=0x7f03a00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:52.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.578+0000 7f03a80e1700 1 -- 172.21.15.152:0/2344665113 shutdown_connections 2024-04-25T01:37:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.578+0000 7f03a80e1700 1 -- 172.21.15.152:0/2344665113 wait complete. 2024-04-25T01:37:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.578+0000 7f03a80e1700 1 Processor -- start 2024-04-25T01:37:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.578+0000 7f03a80e1700 1 -- start start 2024-04-25T01:37:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.578+0000 7f03a80e1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03a0101190 0x7f03a0105120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.578+0000 7f03a80e1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03a0102390 0x7f03a0103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.578+0000 7f03a80e1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0102d30 0x7f03a0103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.578+0000 7f03a80e1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03a010cd60 con 0x7f03a0101190 2024-04-25T01:37:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.578+0000 7f03a80e1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f03a010cbe0 con 0x7f03a0102d30 2024-04-25T01:37:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.578+0000 7f03a80e1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03a010cee0 con 0x7f03a0102390 2024-04-25T01:37:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.578+0000 7f03a667e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0102d30 0x7f03a0103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.578+0000 7f03a667e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0102d30 0x7f03a0103cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45036/0 (socket says 172.21.15.152:45036) 2024-04-25T01:37:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.578+0000 7f03a667e700 1 -- 172.21.15.152:0/1795640889 learned_addr learned my addr 172.21.15.152:0/1795640889 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-25T01:37:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.579+0000 7f03a567c700 1 --2- 172.21.15.152:0/1795640889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03a0102390 0x7f03a0103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:52.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.579+0000 7f03a5e7d700 1 --2- 172.21.15.152:0/1795640889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03a0101190 0x7f03a0105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:52.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.579+0000 7f03a667e700 1 -- 172.21.15.152:0/1795640889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03a0102390 msgr2=0x7f03a0103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:52.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.579+0000 7f03a667e700 1 --2- 172.21.15.152:0/1795640889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03a0102390 0x7f03a0103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:52.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.579+0000 7f03a667e700 1 -- 172.21.15.152:0/1795640889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03a0101190 msgr2=0x7f03a0105120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:52.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.579+0000 7f03a667e700 1 --2- 172.21.15.152:0/1795640889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03a0101190 0x7f03a0105120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:52.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.579+0000 7f03a667e700 1 -- 172.21.15.152:0/1795640889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a0104560 con 0x7f03a0102d30 2024-04-25T01:37:52.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.579+0000 7f03a667e700 1 --2- 172.21.15.152:0/1795640889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0102d30 0x7f03a0103cb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f039800ef30 tx=0x7f039800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:52.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.579+0000 7f0396ffd700 1 -- 172.21.15.152:0/1795640889 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03980092e0 con 0x7f03a0102d30 2024-04-25T01:37:52.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.579+0000 7f0396ffd700 1 -- 172.21.15.152:0/1795640889 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0398004bb0 con 0x7f03a0102d30 2024-04-25T01:37:52.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.579+0000 7f03a80e1700 1 -- 172.21.15.152:0/1795640889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a01a2560 con 0x7f03a0102d30 2024-04-25T01:37:52.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.579+0000 7f0396ffd700 1 -- 172.21.15.152:0/1795640889 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398004280 con 0x7f03a0102d30 2024-04-25T01:37:52.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.579+0000 7f03a80e1700 1 -- 172.21.15.152:0/1795640889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a01a2a20 con 0x7f03a0102d30 2024-04-25T01:37:52.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.580+0000 7f03a80e1700 1 -- 172.21.15.152:0/1795640889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a0068f20 con 0x7f03a0102d30 2024-04-25T01:37:52.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.582+0000 7f0396ffd700 1 -- 172.21.15.152:0/1795640889 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f039801b030 con 0x7f03a0102d30 2024-04-25T01:37:52.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.582+0000 7f0396ffd700 1 --2- 172.21.15.152:0/1795640889 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f038c0a2b90 0x7f038c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:52.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.583+0000 7f0396ffd700 1 -- 172.21.15.152:0/1795640889 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f03980cda10 con 0x7f03a0102d30 2024-04-25T01:37:52.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.583+0000 7f03a5e7d700 1 --2- 172.21.15.152:0/1795640889 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f038c0a2b90 0x7f038c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:52.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.583+0000 7f03a5e7d700 1 --2- 172.21.15.152:0/1795640889 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f038c0a2b90 0x7f038c0a5050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f03a01021f0 tx=0x7f039000a3b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:52.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.586+0000 7f0396ffd700 1 -- 172.21.15.152:0/1795640889 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0398022020 con 0x7f03a0102d30 2024-04-25T01:37:52.743 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:52 smithi078 ceph-mon[29469]: Detected new or changed devices on smithi078 2024-04-25T01:37:52.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:52 smithi078 ceph-mon[29469]: Adjusting osd_memory_target on smithi078 to 17088M 2024-04-25T01:37:52.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:52 smithi078 ceph-mon[29469]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T01:37:52.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:37:52.788+0000 7f03a80e1700 1 -- 172.21.15.152:0/1795640889 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f03a00fe8a0 con 0x7f038c0a2b90 2024-04-25T01:37:52.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:52 smithi196 ceph-mon[32978]: Detected new or changed devices on smithi078 2024-04-25T01:37:52.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:52 smithi196 ceph-mon[32978]: Adjusting osd_memory_target on smithi078 to 17088M 2024-04-25T01:37:52.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:52 smithi196 ceph-mon[32978]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T01:37:53.664 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:53 smithi152 ceph-mon[32465]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:53.664 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:37:53.664 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:37:53.664 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:53.664 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:53 smithi152 ceph-mon[32465]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:37:53.743 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:53 smithi078 ceph-mon[29469]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:53.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:37:53.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:37:53.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:53.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:53 smithi078 ceph-mon[29469]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:37:53.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:53 smithi196 ceph-mon[32978]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:53.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:37:53.853 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:37:53.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:53.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:53 smithi196 ceph-mon[32978]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:37:56.035 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:55 smithi152 ceph-mon[32465]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:37:56.035 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:55 smithi152 ceph-mon[32465]: from='client.? 172.21.15.152:0/2072426160' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54418993-9619-4854-9ce9-4bc5ff572a10"}]: dispatch 2024-04-25T01:37:56.035 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:55 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54418993-9619-4854-9ce9-4bc5ff572a10"}]: dispatch 2024-04-25T01:37:56.036 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:55 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54418993-9619-4854-9ce9-4bc5ff572a10"}]': finished 2024-04-25T01:37:56.036 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:55 smithi152 ceph-mon[32465]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T01:37:56.036 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:55 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:37:56.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:55 smithi078 ceph-mon[29469]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:37:56.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:55 smithi078 ceph-mon[29469]: from='client.? 172.21.15.152:0/2072426160' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54418993-9619-4854-9ce9-4bc5ff572a10"}]: dispatch 2024-04-25T01:37:56.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:55 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54418993-9619-4854-9ce9-4bc5ff572a10"}]: dispatch 2024-04-25T01:37:56.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:55 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54418993-9619-4854-9ce9-4bc5ff572a10"}]': finished 2024-04-25T01:37:56.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:55 smithi078 ceph-mon[29469]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T01:37:56.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:37:56.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:55 smithi196 ceph-mon[32978]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:37:56.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:55 smithi196 ceph-mon[32978]: from='client.? 172.21.15.152:0/2072426160' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54418993-9619-4854-9ce9-4bc5ff572a10"}]: dispatch 2024-04-25T01:37:56.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:55 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54418993-9619-4854-9ce9-4bc5ff572a10"}]: dispatch 2024-04-25T01:37:56.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:55 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54418993-9619-4854-9ce9-4bc5ff572a10"}]': finished 2024-04-25T01:37:56.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:55 smithi196 ceph-mon[32978]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T01:37:56.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:55 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:37:57.103 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:56 smithi196 ceph-mon[32978]: from='client.? 172.21.15.152:0/1417992848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:37:57.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:56 smithi078 ceph-mon[29469]: from='client.? 172.21.15.152:0/1417992848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:37:57.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:56 smithi152 ceph-mon[32465]: from='client.? 172.21.15.152:0/1417992848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:37:58.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:57 smithi078 ceph-mon[29469]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:37:58.285 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:57 smithi152 ceph-mon[32465]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:37:58.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:57 smithi196 ceph-mon[32978]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:00.159 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:37:59 smithi152 ceph-mon[32465]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:00.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:37:59 smithi078 ceph-mon[29469]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:00.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:37:59 smithi196 ceph-mon[32978]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:02.036 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:01 smithi152 ceph-mon[32465]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:02.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:01 smithi078 ceph-mon[29469]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:02.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:01 smithi196 ceph-mon[32978]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:03.180 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T01:38:03.180 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:03.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T01:38:03.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:03.353 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T01:38:03.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:04.160 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:03 smithi152 ceph-mon[32465]: Deploying daemon osd.1 on smithi152 2024-04-25T01:38:04.160 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:03 smithi152 ceph-mon[32465]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:04.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:03 smithi078 ceph-mon[29469]: Deploying daemon osd.1 on smithi152 2024-04-25T01:38:04.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:03 smithi078 ceph-mon[29469]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:04.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:03 smithi196 ceph-mon[32978]: Deploying daemon osd.1 on smithi152 2024-04-25T01:38:04.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:03 smithi196 ceph-mon[32978]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:06.085 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:05 smithi152 ceph-mon[32465]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:06.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:05 smithi078 ceph-mon[29469]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:06.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:05 smithi196 ceph-mon[32978]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:08.202 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:07 smithi152 ceph-mon[32465]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:08.202 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:07 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:08.202 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:07 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:08.202 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:07 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:08.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:07 smithi078 ceph-mon[29469]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:08.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:07 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:08.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:07 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:08.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:07 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:08.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:07 smithi196 ceph-mon[32978]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:08.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:07 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:08.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:07 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:08.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:07 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:09.976 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:09 smithi152 ceph-mon[32465]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:10.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:09 smithi078 ceph-mon[29469]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:10.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:09 smithi196 ceph-mon[32978]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:10.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:38:11.007+0000 7f0396ffd700 1 -- 172.21.15.152:0/1795640889 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f03a00fe8a0 con 0x7f038c0a2b90 2024-04-25T01:38:10.987 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 1 on host 'smithi152' 2024-04-25T01:38:10.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:38:11.009+0000 7f03a80e1700 1 -- 172.21.15.152:0/1795640889 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f038c0a2b90 msgr2=0x7f038c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:10.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:38:11.009+0000 7f03a80e1700 1 --2- 172.21.15.152:0/1795640889 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f038c0a2b90 0x7f038c0a5050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f03a01021f0 tx=0x7f039000a3b0 comp rx=0 tx=0).stop 2024-04-25T01:38:10.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:38:11.010+0000 7f03a80e1700 1 -- 172.21.15.152:0/1795640889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0102d30 msgr2=0x7f03a0103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:10.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:38:11.010+0000 7f03a80e1700 1 --2- 172.21.15.152:0/1795640889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0102d30 0x7f03a0103cb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f039800ef30 tx=0x7f039800c3a0 comp rx=0 tx=0).stop 2024-04-25T01:38:10.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:38:11.010+0000 7f03a80e1700 1 -- 172.21.15.152:0/1795640889 shutdown_connections 2024-04-25T01:38:10.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:38:11.010+0000 7f03a80e1700 1 --2- 172.21.15.152:0/1795640889 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f038c0a2b90 0x7f038c0a5050 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:10.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:38:11.010+0000 7f03a80e1700 1 --2- 172.21.15.152:0/1795640889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03a0102d30 0x7f03a0103cb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:10.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:38:11.010+0000 7f03a80e1700 1 --2- 172.21.15.152:0/1795640889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03a0101190 0x7f03a0105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:10.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:38:11.010+0000 7f03a80e1700 1 --2- 172.21.15.152:0/1795640889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03a0102390 0x7f03a0103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:10.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:38:11.010+0000 7f03a80e1700 1 -- 172.21.15.152:0/1795640889 >> 172.21.15.152:0/1795640889 conn(0x7f03a00fc730 msgr2=0x7f03a00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:10.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:38:11.010+0000 7f03a80e1700 1 -- 172.21.15.152:0/1795640889 shutdown_connections 2024-04-25T01:38:10.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:38:11.010+0000 7f03a80e1700 1 -- 172.21.15.152:0/1795640889 wait complete. 2024-04-25T01:38:11.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:11 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:11 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:11 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:11.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:11 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:11.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:11 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:11 smithi152 ceph-mon[32465]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:11.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:11 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:11 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:11 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:11.569 DEBUG:teuthology.orchestra.run.smithi152:osd.1> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T01:38:11.572 INFO:tasks.cephadm:Deploying osd.2 on smithi196 with /dev/vg_nvme/lv_4... 2024-04-25T01:38:11.572 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T01:38:11.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:11 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:11 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:11 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:11.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:11 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:11.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:11 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:11 smithi078 ceph-mon[29469]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:11.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:11 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:11 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:11 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:11.813 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:38:11.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:11 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:11 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:11 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:11.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:11 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:11.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:11 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:11 smithi196 ceph-mon[32978]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:11.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:11 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:11 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:11.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:11 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:12 smithi196 ceph-mon[32978]: from='osd.1 [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:38:12.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:12 smithi196 ceph-mon[32978]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:38:12.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:12 smithi078 ceph-mon[29469]: from='osd.1 [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:38:12.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:12 smithi078 ceph-mon[29469]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:38:12.763 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:12 smithi152 ceph-mon[32465]: from='osd.1 [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:38:12.763 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:12 smithi152 ceph-mon[32465]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:38:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:13 smithi196 ceph-mon[32978]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T01:38:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:13 smithi196 ceph-mon[32978]: from='osd.1 [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:38:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:13 smithi196 ceph-mon[32978]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T01:38:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:13 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:13.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:13 smithi196 ceph-mon[32978]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:38:13.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:13 smithi196 ceph-mon[32978]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:13.725 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T01:38:13.725 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T01:38:13.725 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-04-25T01:38:13.725 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-04-25T01:38:13.725 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113135 s, 927 MB/s 2024-04-25T01:38:13.725 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-04-25T01:38:13.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:13 smithi078 ceph-mon[29469]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T01:38:13.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:13 smithi078 ceph-mon[29469]: from='osd.1 [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:38:13.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:13 smithi078 ceph-mon[29469]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T01:38:13.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:13 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:13.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:13 smithi078 ceph-mon[29469]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:38:13.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:13 smithi078 ceph-mon[29469]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:13.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:13 smithi152 ceph-mon[32465]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T01:38:13.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:13 smithi152 ceph-mon[32465]: from='osd.1 [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:38:13.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:13 smithi152 ceph-mon[32465]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T01:38:13.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:13 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:13.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:13 smithi152 ceph-mon[32465]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:38:13.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:13 smithi152 ceph-mon[32465]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:13.787 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:38:13 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-1[37052]: 2024-04-25T01:38:13.415+0000 7fdfb5967700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T01:38:13.787 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:38:13 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-1[37052]: 2024-04-25T01:38:13.420+0000 7fdfb175a700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:38:14.360 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-04-25T01:38:14.505 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:14 smithi152 ceph-mon[32465]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T01:38:14.506 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:14 smithi152 ceph-mon[32465]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T01:38:14.506 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:14 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:14.506 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:14 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:14.506 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:14 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:14.601 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:38:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:14 smithi196 ceph-mon[32978]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T01:38:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:14 smithi196 ceph-mon[32978]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T01:38:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:14.604 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:14.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:14 smithi078 ceph-mon[29469]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T01:38:14.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:14 smithi078 ceph-mon[29469]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T01:38:14.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:14.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:14.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: purged_snaps scrub starts 2024-04-25T01:38:15.723 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: purged_snaps scrub ok 2024-04-25T01:38:15.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: osd.1 [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494] boot 2024-04-25T01:38:15.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T01:38:15.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:15.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: Detected new or changed devices on smithi152 2024-04-25T01:38:15.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:15.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:15.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:38:15.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: Adjusting osd_memory_target on smithi152 to 17088M 2024-04-25T01:38:15.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:15.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:15.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:15.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:15.724 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:15 smithi196 ceph-mon[32978]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:15.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: purged_snaps scrub starts 2024-04-25T01:38:15.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: purged_snaps scrub ok 2024-04-25T01:38:15.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: osd.1 [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494] boot 2024-04-25T01:38:15.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T01:38:15.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:15.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: Detected new or changed devices on smithi152 2024-04-25T01:38:15.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:15.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:15.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:38:15.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: Adjusting osd_memory_target on smithi152 to 17088M 2024-04-25T01:38:15.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:15.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:15.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:15.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:15.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:15 smithi078 ceph-mon[29469]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:15.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: purged_snaps scrub starts 2024-04-25T01:38:15.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: purged_snaps scrub ok 2024-04-25T01:38:15.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: osd.1 [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494] boot 2024-04-25T01:38:15.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T01:38:15.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:38:15.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: Detected new or changed devices on smithi152 2024-04-25T01:38:15.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:15.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:15.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:38:15.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: Adjusting osd_memory_target on smithi152 to 17088M 2024-04-25T01:38:15.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:15.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:15.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:15.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:15.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:15 smithi152 ceph-mon[32465]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:38:16.214 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.232+0000 7f17f6808700 1 -- 172.21.15.196:0/752639568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0102390 msgr2=0x7f17f01027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:16.214 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.232+0000 7f17f6808700 1 --2- 172.21.15.196:0/752639568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0102390 0x7f17f01027f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009990 tx=0x7f17e4009ca0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.214 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.232+0000 7f17f6808700 1 -- 172.21.15.196:0/752639568 shutdown_connections 2024-04-25T01:38:16.214 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.232+0000 7f17f6808700 1 --2- 172.21.15.196:0/752639568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17f0102d30 0x7f17f01095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.214 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.232+0000 7f17f6808700 1 --2- 172.21.15.196:0/752639568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0101190 0x7f17f01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.214 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.232+0000 7f17f6808700 1 --2- 172.21.15.196:0/752639568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0102390 0x7f17f01027f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.232+0000 7f17f6808700 1 -- 172.21.15.196:0/752639568 >> 172.21.15.196:0/752639568 conn(0x7f17f00fc730 msgr2=0x7f17f00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:16.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.232+0000 7f17f6808700 1 -- 172.21.15.196:0/752639568 shutdown_connections 2024-04-25T01:38:16.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.232+0000 7f17f6808700 1 -- 172.21.15.196:0/752639568 wait complete. 2024-04-25T01:38:16.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.232+0000 7f17f6808700 1 Processor -- start 2024-04-25T01:38:16.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.232+0000 7f17f6808700 1 -- start start 2024-04-25T01:38:16.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17f6808700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0101190 0x7f17f0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:16.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17f6808700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0102390 0x7f17f0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:16.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17f6808700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17f0102d30 0x7f17f019fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:16.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17f6808700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17f010cd80 con 0x7f17f0102390 2024-04-25T01:38:16.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17f6808700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f17f010cc00 con 0x7f17f0102d30 2024-04-25T01:38:16.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17f6808700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17f010cf00 con 0x7f17f0101190 2024-04-25T01:38:16.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17effff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0101190 0x7f17f0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:16.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17effff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0101190 0x7f17f0198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:51732/0 (socket says 172.21.15.196:51732) 2024-04-25T01:38:16.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17effff700 1 -- 172.21.15.196:0/3788504710 learned_addr learned my addr 172.21.15.196:0/3788504710 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T01:38:16.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17f4da5700 1 --2- 172.21.15.196:0/3788504710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17f0102d30 0x7f17f019fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:16.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17ef7fe700 1 --2- 172.21.15.196:0/3788504710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0102390 0x7f17f0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:16.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17effff700 1 -- 172.21.15.196:0/3788504710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17f0102d30 msgr2=0x7f17f019fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:16.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17effff700 1 --2- 172.21.15.196:0/3788504710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17f0102d30 0x7f17f019fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17effff700 1 -- 172.21.15.196:0/3788504710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0102390 msgr2=0x7f17f0198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:16.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17effff700 1 --2- 172.21.15.196:0/3788504710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0102390 0x7f17f0198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17effff700 1 -- 172.21.15.196:0/3788504710 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f01a0170 con 0x7f17f0101190 2024-04-25T01:38:16.216 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17effff700 1 --2- 172.21.15.196:0/3788504710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0101190 0x7f17f0198510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f17e000d9b0 tx=0x7f17e000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:16.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17ed7fa700 1 -- 172.21.15.196:0/3788504710 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0017030 con 0x7f17f0101190 2024-04-25T01:38:16.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17ed7fa700 1 -- 172.21.15.196:0/3788504710 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e0004500 con 0x7f17f0101190 2024-04-25T01:38:16.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17f6808700 1 -- 172.21.15.196:0/3788504710 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f01a0460 con 0x7f17f0101190 2024-04-25T01:38:16.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17ed7fa700 1 -- 172.21.15.196:0/3788504710 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0003aa0 con 0x7f17f0101190 2024-04-25T01:38:16.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.233+0000 7f17f6808700 1 -- 172.21.15.196:0/3788504710 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f01a09a0 con 0x7f17f0101190 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.234+0000 7f17f6808700 1 -- 172.21.15.196:0/3788504710 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f0068f20 con 0x7f17f0101190 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.235+0000 7f17ed7fa700 1 -- 172.21.15.196:0/3788504710 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f17e001a070 con 0x7f17f0101190 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.236+0000 7f17ed7fa700 1 --2- 172.21.15.196:0/3788504710 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17d80a2b90 0x7f17d80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.236+0000 7f17ed7fa700 1 -- 172.21.15.196:0/3788504710 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f17e00cd4b0 con 0x7f17f0101190 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.236+0000 7f17ef7fe700 1 --2- 172.21.15.196:0/3788504710 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17d80a2b90 0x7f17d80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.237+0000 7f17ef7fe700 1 --2- 172.21.15.196:0/3788504710 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17d80a2b90 0x7f17d80a5050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f17e4003a60 tx=0x7f17e400b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:16.220 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.240+0000 7f17ed7fa700 1 -- 172.21.15.196:0/3788504710 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f17e000e350 con 0x7f17f0101190 2024-04-25T01:38:16.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:16.443+0000 7f17f6808700 1 -- 172.21.15.196:0/3788504710 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f17f006a620 con 0x7f17d80a2b90 2024-04-25T01:38:17.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:17 smithi196 ceph-mon[32978]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T01:38:17.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:17 smithi196 ceph-mon[32978]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:17.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:38:17.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:38:17.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:17.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:17 smithi196 ceph-mon[32978]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:17.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:17 smithi078 ceph-mon[29469]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T01:38:17.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:17 smithi078 ceph-mon[29469]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:17.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:38:17.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:38:17.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:17.745 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:17 smithi078 ceph-mon[29469]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:17 smithi152 ceph-mon[32465]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T01:38:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:17 smithi152 ceph-mon[32465]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:38:17.786 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:38:17.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:17.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:17 smithi152 ceph-mon[32465]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:20.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:19 smithi078 ceph-mon[29469]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:20.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:19 smithi078 ceph-mon[29469]: from='client.? 172.21.15.196:0/1047028185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcc6ae83-6a79-4c84-bf51-d4c6a1c15630"}]: dispatch 2024-04-25T01:38:20.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:19 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcc6ae83-6a79-4c84-bf51-d4c6a1c15630"}]: dispatch 2024-04-25T01:38:20.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:19 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcc6ae83-6a79-4c84-bf51-d4c6a1c15630"}]': finished 2024-04-25T01:38:20.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:19 smithi078 ceph-mon[29469]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T01:38:20.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:20.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:19 smithi078 ceph-mon[29469]: from='client.? 172.21.15.196:0/2000656456' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:38:20.286 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:19 smithi152 ceph-mon[32465]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:20.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:19 smithi152 ceph-mon[32465]: from='client.? 172.21.15.196:0/1047028185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcc6ae83-6a79-4c84-bf51-d4c6a1c15630"}]: dispatch 2024-04-25T01:38:20.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:19 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcc6ae83-6a79-4c84-bf51-d4c6a1c15630"}]: dispatch 2024-04-25T01:38:20.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:19 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcc6ae83-6a79-4c84-bf51-d4c6a1c15630"}]': finished 2024-04-25T01:38:20.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:19 smithi152 ceph-mon[32465]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T01:38:20.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:20.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:19 smithi152 ceph-mon[32465]: from='client.? 172.21.15.196:0/2000656456' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:38:20.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:19 smithi196 ceph-mon[32978]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:20.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:19 smithi196 ceph-mon[32978]: from='client.? 172.21.15.196:0/1047028185' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcc6ae83-6a79-4c84-bf51-d4c6a1c15630"}]: dispatch 2024-04-25T01:38:20.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:19 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcc6ae83-6a79-4c84-bf51-d4c6a1c15630"}]: dispatch 2024-04-25T01:38:20.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:19 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcc6ae83-6a79-4c84-bf51-d4c6a1c15630"}]': finished 2024-04-25T01:38:20.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:19 smithi196 ceph-mon[32978]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T01:38:20.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:20.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:19 smithi196 ceph-mon[32978]: from='client.? 172.21.15.196:0/2000656456' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:38:22.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:21 smithi078 ceph-mon[29469]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:22.286 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:21 smithi152 ceph-mon[32465]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:22.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:21 smithi196 ceph-mon[32978]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:24.168 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:23 smithi196 ceph-mon[32978]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:24.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:23 smithi078 ceph-mon[29469]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:24.286 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:23 smithi152 ceph-mon[32465]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:26.128 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:25 smithi196 ceph-mon[32978]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:26.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:25 smithi078 ceph-mon[29469]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:26.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:25 smithi152 ceph-mon[32465]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:27.073 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:26 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T01:38:27.073 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:26 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:27.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:26 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T01:38:27.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:26 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:27.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:26 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T01:38:27.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:26 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:27.951 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:27 smithi196 ceph-mon[32978]: Deploying daemon osd.2 on smithi196 2024-04-25T01:38:27.952 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:27 smithi196 ceph-mon[32978]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:28.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:27 smithi078 ceph-mon[29469]: Deploying daemon osd.2 on smithi196 2024-04-25T01:38:28.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:27 smithi078 ceph-mon[29469]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:28.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:27 smithi152 ceph-mon[32465]: Deploying daemon osd.2 on smithi196 2024-04-25T01:38:28.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:27 smithi152 ceph-mon[32465]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:29.995 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:29 smithi196 ceph-mon[32978]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:30.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:29 smithi078 ceph-mon[29469]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:30.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:29 smithi152 ceph-mon[32465]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:32.144 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:31 smithi196 ceph-mon[32978]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:32.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:31 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:32.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:31 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:32.145 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:31 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:32.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:31 smithi078 ceph-mon[29469]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:32.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:31 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:32.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:31 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:32.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:31 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:32.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:31 smithi152 ceph-mon[32465]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:32.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:31 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:32.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:31 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:32.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:31 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:34.155 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:33 smithi196 ceph-mon[32978]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:34.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:33 smithi078 ceph-mon[29469]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:34.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:33 smithi152 ceph-mon[32465]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:35.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:35.059+0000 7f17ed7fa700 1 -- 172.21.15.196:0/3788504710 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f17f006a620 con 0x7f17d80a2b90 2024-04-25T01:38:35.039 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 2 on host 'smithi196' 2024-04-25T01:38:35.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:35.061+0000 7f17f6808700 1 -- 172.21.15.196:0/3788504710 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17d80a2b90 msgr2=0x7f17d80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:35.061+0000 7f17f6808700 1 --2- 172.21.15.196:0/3788504710 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17d80a2b90 0x7f17d80a5050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f17e4003a60 tx=0x7f17e400b540 comp rx=0 tx=0).stop 2024-04-25T01:38:35.041 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:35.061+0000 7f17f6808700 1 -- 172.21.15.196:0/3788504710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0101190 msgr2=0x7f17f0198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:35.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:35.061+0000 7f17f6808700 1 --2- 172.21.15.196:0/3788504710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0101190 0x7f17f0198510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f17e000d9b0 tx=0x7f17e000dcc0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:35.061+0000 7f17f6808700 1 -- 172.21.15.196:0/3788504710 shutdown_connections 2024-04-25T01:38:35.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:35.061+0000 7f17f6808700 1 --2- 172.21.15.196:0/3788504710 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17d80a2b90 0x7f17d80a5050 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:35.061+0000 7f17f6808700 1 --2- 172.21.15.196:0/3788504710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17f0101190 0x7f17f0198510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:35.061+0000 7f17f6808700 1 --2- 172.21.15.196:0/3788504710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17f0102d30 0x7f17f019fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:35.061+0000 7f17f6808700 1 --2- 172.21.15.196:0/3788504710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f0102390 0x7f17f0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:35.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:35.061+0000 7f17f6808700 1 -- 172.21.15.196:0/3788504710 >> 172.21.15.196:0/3788504710 conn(0x7f17f00fc730 msgr2=0x7f17f0069f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:35.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:35.061+0000 7f17f6808700 1 -- 172.21.15.196:0/3788504710 shutdown_connections 2024-04-25T01:38:35.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:38:35.061+0000 7f17f6808700 1 -- 172.21.15.196:0/3788504710 wait complete. 2024-04-25T01:38:35.680 DEBUG:teuthology.orchestra.run.smithi196:osd.2> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T01:38:35.683 INFO:tasks.cephadm:Deploying osd.3 on smithi078 with /dev/vg_nvme/lv_3... 2024-04-25T01:38:35.683 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T01:38:35.927 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:38:35.995 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:35 smithi078 ceph-mon[29469]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:35.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:35 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:35.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:35 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:35.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:35 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:35.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:35 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:35.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:35 smithi078 ceph-mon[29469]: from='osd.2 [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:38:35.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:35 smithi078 ceph-mon[29469]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:38:36.007 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:35 smithi196 ceph-mon[32978]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:36.007 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:35 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:36.007 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:35 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:36.007 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:35 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:36.007 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:35 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:36.007 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:35 smithi196 ceph-mon[32978]: from='osd.2 [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:38:36.008 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:35 smithi196 ceph-mon[32978]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:38:36.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:35 smithi152 ceph-mon[32465]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:36.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:35 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:36.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:35 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:36.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:35 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:36.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:35 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:36.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:35 smithi152 ceph-mon[32465]: from='osd.2 [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:38:36.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:35 smithi152 ceph-mon[32465]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:38:37.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:37 smithi078 ceph-mon[29469]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T01:38:37.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:37 smithi078 ceph-mon[29469]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T01:38:37.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:37.344 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:37 smithi078 ceph-mon[29469]: from='osd.2 [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:38:37.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:37 smithi078 ceph-mon[29469]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:38:37.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:37 smithi078 ceph-mon[29469]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:37.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:37 smithi196 ceph-mon[32978]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T01:38:37.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:37 smithi196 ceph-mon[32978]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T01:38:37.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:37.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:37 smithi196 ceph-mon[32978]: from='osd.2 [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:38:37.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:37 smithi196 ceph-mon[32978]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:38:37.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:37 smithi196 ceph-mon[32978]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:37.356 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:38:37 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-2[38112]: 2024-04-25T01:38:37.111+0000 7f9e4e0cf700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T01:38:37.356 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:38:37 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-2[38112]: 2024-04-25T01:38:37.115+0000 7f9e49ec2700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:38:37.537 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:37 smithi152 ceph-mon[32465]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T01:38:37.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:37 smithi152 ceph-mon[32465]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T01:38:37.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:37.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:37 smithi152 ceph-mon[32465]: from='osd.2 [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:38:37.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:37 smithi152 ceph-mon[32465]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:38:37.539 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:37 smithi152 ceph-mon[32465]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:37.914 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T01:38:37.915 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T01:38:37.915 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-25T01:38:37.915 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-25T01:38:37.915 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114834 s, 913 MB/s 2024-04-25T01:38:37.915 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-25T01:38:38.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:38 smithi196 ceph-mon[32978]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-25T01:38:38.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:38 smithi196 ceph-mon[32978]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T01:38:38.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:38 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:38.273 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:38 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:38.362 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:38 smithi152 ceph-mon[32465]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-25T01:38:38.362 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:38 smithi152 ceph-mon[32465]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T01:38:38.362 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:38 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:38.362 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:38 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:38.495 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:38 smithi078 ceph-mon[29469]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-25T01:38:38.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:38 smithi078 ceph-mon[29469]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T01:38:38.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:38 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:38.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:38 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:38.589 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_3 2024-04-25T01:38:38.839 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:38:39.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: purged_snaps scrub starts 2024-04-25T01:38:39.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: purged_snaps scrub ok 2024-04-25T01:38:39.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: osd.2 [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723] boot 2024-04-25T01:38:39.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T01:38:39.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:39.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:39.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:39.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:38:39.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:39.217 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:39.217 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:39.217 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:39.217 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:39.217 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:39.217 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:39.217 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:39 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:39.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: purged_snaps scrub starts 2024-04-25T01:38:39.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: purged_snaps scrub ok 2024-04-25T01:38:39.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: osd.2 [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723] boot 2024-04-25T01:38:39.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T01:38:39.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:39.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:39.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:39.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:38:39.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:39.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:39.539 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:39.539 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:39.539 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:39.539 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:39.539 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:39.539 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:39 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:39.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: purged_snaps scrub starts 2024-04-25T01:38:39.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: purged_snaps scrub ok 2024-04-25T01:38:39.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: osd.2 [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723] boot 2024-04-25T01:38:39.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T01:38:39.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:38:39.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:39.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:39.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:38:39.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:39.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:39.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:39.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:39.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:39.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:39.607 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:39.607 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:39 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:40.181 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:40 smithi078 ceph-mon[29469]: Detected new or changed devices on smithi196 2024-04-25T01:38:40.181 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:40 smithi078 ceph-mon[29469]: Adjusting osd_memory_target on smithi196 to 17088M 2024-04-25T01:38:40.181 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:40 smithi078 ceph-mon[29469]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:40.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:40 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:38:40.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:40 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T01:38:40.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:40 smithi078 ceph-mon[29469]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T01:38:40.182 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:40 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:38:40.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.430+0000 7fe52c802700 1 -- 172.21.15.78:0/2168528131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5281021d0 msgr2=0x7fe5281025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:40.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.430+0000 7fe52c802700 1 --2- 172.21.15.78:0/2168528131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5281021d0 0x7fe5281025f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe5180099c0 tx=0x7fe518009cd0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.430+0000 7fe52c802700 1 -- 172.21.15.78:0/2168528131 shutdown_connections 2024-04-25T01:38:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.430+0000 7fe52c802700 1 --2- 172.21.15.78:0/2168528131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe528103d70 0x7fe52810a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.430+0000 7fe52c802700 1 --2- 172.21.15.78:0/2168528131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5281021d0 0x7fe5281025f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.430+0000 7fe52c802700 1 --2- 172.21.15.78:0/2168528131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5281033d0 0x7fe528103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.430+0000 7fe52c802700 1 -- 172.21.15.78:0/2168528131 >> 172.21.15.78:0/2168528131 conn(0x7fe5280fd770 msgr2=0x7fe5280ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.430+0000 7fe52c802700 1 -- 172.21.15.78:0/2168528131 shutdown_connections 2024-04-25T01:38:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.430+0000 7fe52c802700 1 -- 172.21.15.78:0/2168528131 wait complete. 2024-04-25T01:38:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.430+0000 7fe52c802700 1 Processor -- start 2024-04-25T01:38:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52c802700 1 -- start start 2024-04-25T01:38:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52c802700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5281021d0 0x7fe528198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52c802700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5281033d0 0x7fe528198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52c802700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe528103d70 0x7fe52819f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52c802700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe52810de20 con 0x7fe528103d70 2024-04-25T01:38:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52c802700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe52810dca0 con 0x7fe5281021d0 2024-04-25T01:38:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52c802700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe52810dfa0 con 0x7fe5281033d0 2024-04-25T01:38:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe528103d70 0x7fe52819f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52759e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe528103d70 0x7fe52819f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34990/0 (socket says 172.21.15.78:34990) 2024-04-25T01:38:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52759e700 1 -- 172.21.15.78:0/592216339 learned_addr learned my addr 172.21.15.78:0/592216339 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:38:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe526d9d700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5281021d0 0x7fe528198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52659c700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5281033d0 0x7fe528198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52759e700 1 -- 172.21.15.78:0/592216339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5281033d0 msgr2=0x7fe528198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52759e700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5281033d0 0x7fe528198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52759e700 1 -- 172.21.15.78:0/592216339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5281021d0 msgr2=0x7fe528198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52759e700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5281021d0 0x7fe528198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52759e700 1 -- 172.21.15.78:0/592216339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5281a0060 con 0x7fe528103d70 2024-04-25T01:38:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe526d9d700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5281021d0 0x7fe528198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:38:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.431+0000 7fe52759e700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe528103d70 0x7fe52819f9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe52000b9a0 tx=0x7fe52000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.432+0000 7fe50ffff700 1 -- 172.21.15.78:0/592216339 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52000d850 con 0x7fe528103d70 2024-04-25T01:38:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.432+0000 7fe50ffff700 1 -- 172.21.15.78:0/592216339 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe52000e420 con 0x7fe528103d70 2024-04-25T01:38:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.432+0000 7fe52c802700 1 -- 172.21.15.78:0/592216339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5281a0350 con 0x7fe528103d70 2024-04-25T01:38:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.432+0000 7fe50ffff700 1 -- 172.21.15.78:0/592216339 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520017660 con 0x7fe528103d70 2024-04-25T01:38:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.432+0000 7fe52c802700 1 -- 172.21.15.78:0/592216339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5281a0910 con 0x7fe528103d70 2024-04-25T01:38:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.433+0000 7fe52c802700 1 -- 172.21.15.78:0/592216339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe528068f20 con 0x7fe528103d70 2024-04-25T01:38:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.434+0000 7fe50ffff700 1 -- 172.21.15.78:0/592216339 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fe52000e5d0 con 0x7fe528103d70 2024-04-25T01:38:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.435+0000 7fe50ffff700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe5100a2b90 0x7fe5100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.435+0000 7fe526d9d700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe5100a2b90 0x7fe5100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.435+0000 7fe50ffff700 1 -- 172.21.15.78:0/592216339 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7fe5200cdae0 con 0x7fe528103d70 2024-04-25T01:38:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.435+0000 7fe526d9d700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe5100a2b90 0x7fe5100a5050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe518003890 tx=0x7fe518005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.439+0000 7fe50ffff700 1 -- 172.21.15.78:0/592216339 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe520013030 con 0x7fe528103d70 2024-04-25T01:38:40.537 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:40 smithi152 ceph-mon[32465]: Detected new or changed devices on smithi196 2024-04-25T01:38:40.537 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:40 smithi152 ceph-mon[32465]: Adjusting osd_memory_target on smithi196 to 17088M 2024-04-25T01:38:40.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:40 smithi152 ceph-mon[32465]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:40.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:40 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:38:40.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:40 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T01:38:40.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:40 smithi152 ceph-mon[32465]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T01:38:40.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:40 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:38:40.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:40 smithi196 ceph-mon[32978]: Detected new or changed devices on smithi196 2024-04-25T01:38:40.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:40 smithi196 ceph-mon[32978]: Adjusting osd_memory_target on smithi196 to 17088M 2024-04-25T01:38:40.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:40 smithi196 ceph-mon[32978]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T01:38:40.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:40 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:38:40.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:40 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T01:38:40.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:40 smithi196 ceph-mon[32978]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T01:38:40.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:40 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:38:40.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:40.653+0000 7fe52c802700 1 -- 172.21.15.78:0/592216339 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe528062670 con 0x7fe5100a2b90 2024-04-25T01:38:40.743 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:38:40 smithi078 sudo[46198]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:38:40.743 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:38:40 smithi078 sudo[46198]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:38:40.743 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:38:40 smithi078 sudo[46198]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:38:40.743 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:38:40 smithi078 sudo[46198]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:38:40.995 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:38:40 smithi078 sudo[46268]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:38:40.995 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:38:40 smithi078 sudo[46268]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:38:40.996 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:38:40 smithi078 sudo[46268]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:38:40.996 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:38:40 smithi078 sudo[46268]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:38:41.262 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:38:41 smithi152 sudo[41184]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:38:41.263 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:38:41 smithi152 sudo[41184]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:38:41.263 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:38:41 smithi152 sudo[41184]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:38:41.263 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:38:41 smithi152 sudo[41184]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:38:41.263 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:41 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T01:38:41.263 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:41 smithi152 ceph-mon[32465]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T01:38:41.263 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:41 smithi152 ceph-mon[32465]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:41.263 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:41 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:38:41.263 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:41 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:38:41.263 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:41 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:41.264 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:41 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:41.264 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:41 smithi152 ceph-mon[32465]: pgmap v82: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:38:41.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T01:38:41.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:41 smithi078 ceph-mon[29469]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T01:38:41.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:41 smithi078 ceph-mon[29469]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:41.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:38:41.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:38:41.385 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:41.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:41.386 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:41 smithi078 ceph-mon[29469]: pgmap v82: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:38:41.531 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:41 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T01:38:41.531 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:41 smithi196 ceph-mon[32978]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T01:38:41.531 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:41 smithi196 ceph-mon[32978]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:41.531 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:41 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:38:41.531 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:41 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:38:41.531 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:41 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:41.531 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:41 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:41.531 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:41 smithi196 ceph-mon[32978]: pgmap v82: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:38:41.537 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:38:41 smithi152 sudo[41187]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:38:41.537 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:38:41 smithi152 sudo[41187]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:38:41.537 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:38:41 smithi152 sudo[41187]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:38:41.537 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:38:41 smithi152 sudo[41187]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:38:41.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:41.797+0000 7fe50ffff700 1 -- 172.21.15.78:0/592216339 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe520097e60 con 0x7fe528103d70 2024-04-25T01:38:41.783 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:38:41 smithi196 sudo[42150]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:38:41.783 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:38:41 smithi196 sudo[42150]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:38:41.783 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:38:41 smithi196 sudo[42150]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:38:41.783 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:38:41 smithi196 sudo[42150]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:38:42.105 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:38:41 smithi196 sudo[42153]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:38:42.105 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:38:41 smithi196 sudo[42153]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:38:42.105 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:38:41 smithi196 sudo[42153]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:38:42.105 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:38:41 smithi196 sudo[42153]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:38:42.312 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:42 smithi078 sudo[46347]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:38:42.312 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:42 smithi078 sudo[46347]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:38:42.312 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:42 smithi078 sudo[46347]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:38:42.312 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:42 smithi078 sudo[46347]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:38:42.570 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:42 smithi196 ceph-mon[32978]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T01:38:42.570 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:42 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:42.570 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:42 smithi196 ceph-mon[32978]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T01:38:42.570 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:42 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:42.570 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:42 smithi196 ceph-mon[32978]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:38:42.570 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:42 smithi196 sudo[42156]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:38:42.570 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:42 smithi196 sudo[42156]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:38:42.571 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:42 smithi196 sudo[42156]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:38:42.571 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:42 smithi196 sudo[42156]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:38:42.659 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:42 smithi078 sudo[46371]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:38:42.660 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:42 smithi078 sudo[46371]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:38:42.660 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:42 smithi078 sudo[46371]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:38:42.660 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:42 smithi078 sudo[46371]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:38:42.660 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:42 smithi078 ceph-mon[29469]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T01:38:42.660 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:42 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:42.660 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:42 smithi078 ceph-mon[29469]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T01:38:42.660 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:42 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:42.660 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:42 smithi078 ceph-mon[29469]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:38:42.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:42 smithi152 ceph-mon[32465]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T01:38:42.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:42 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:42.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:42 smithi152 ceph-mon[32465]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T01:38:42.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:42 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:42.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:42 smithi152 ceph-mon[32465]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:38:42.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:42 smithi196 sudo[42159]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:38:42.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:42 smithi196 sudo[42159]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:38:42.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:42 smithi196 sudo[42159]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:38:42.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:42 smithi196 sudo[42159]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:38:43.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 sudo[41190]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:38:43.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 sudo[41190]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:38:43.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 sudo[41190]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:38:43.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 sudo[41190]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:38:43.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:38:43.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:38:43.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:38:43.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:38:43.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:38:43.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:38:43.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:38:43.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:38:43.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:38:43.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:38:43.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:38:43.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:38:43.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:38:43.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:38:43.607 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:43 smithi196 ceph-mon[32978]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:38:43.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:38:43.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:38:43.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:38:43.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:38:43.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:38:43.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:38:43.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:38:43.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:38:43.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:38:43.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:38:43.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:38:43.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:38:43.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:38:43.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:38:43.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:43 smithi078 ceph-mon[29469]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:38:43.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 sudo[41193]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:38:43.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 sudo[41193]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:38:43.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 sudo[41193]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:38:43.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 sudo[41193]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:38:43.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:38:43.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:38:43.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:38:43.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:38:43.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:38:43.789 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:38:43.789 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:38:43.789 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:38:43.789 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:38:43.789 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:38:43.789 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T01:38:43.789 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T01:38:43.789 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T01:38:43.789 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:38:43.789 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:43 smithi152 ceph-mon[32465]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:38:44.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:44 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/4138697104' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "065344a4-91bb-4940-821e-1d7abd0dc292"}]: dispatch 2024-04-25T01:38:44.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:44 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/4138697104' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "065344a4-91bb-4940-821e-1d7abd0dc292"}]': finished 2024-04-25T01:38:44.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:44 smithi196 ceph-mon[32978]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T01:38:44.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:44 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:38:44.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:44 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/176144762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:38:44.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:44 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/4138697104' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "065344a4-91bb-4940-821e-1d7abd0dc292"}]: dispatch 2024-04-25T01:38:44.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:44 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/4138697104' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "065344a4-91bb-4940-821e-1d7abd0dc292"}]': finished 2024-04-25T01:38:44.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:44 smithi078 ceph-mon[29469]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T01:38:44.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:44 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:38:44.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:44 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/176144762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:38:44.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:44 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/4138697104' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "065344a4-91bb-4940-821e-1d7abd0dc292"}]: dispatch 2024-04-25T01:38:44.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:44 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/4138697104' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "065344a4-91bb-4940-821e-1d7abd0dc292"}]': finished 2024-04-25T01:38:44.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:44 smithi152 ceph-mon[32465]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T01:38:44.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:44 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:38:44.788 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:44 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/176144762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:38:45.605 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:45 smithi196 ceph-mon[32978]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:38:45.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:45 smithi078 ceph-mon[29469]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:38:45.787 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:45 smithi152 ceph-mon[32465]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:38:47.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:47 smithi078 ceph-mon[29469]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-04-25T01:38:48.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:47 smithi152 ceph-mon[32465]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-04-25T01:38:48.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:47 smithi196 ceph-mon[32978]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-04-25T01:38:50.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:49 smithi078 ceph-mon[29469]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-04-25T01:38:50.287 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:49 smithi152 ceph-mon[32465]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-04-25T01:38:50.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:49 smithi196 ceph-mon[32978]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-04-25T01:38:50.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:50 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T01:38:50.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:50 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:51.038 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:50 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T01:38:51.038 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:50 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:51.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:50 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T01:38:51.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:50 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:52.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:51 smithi078 ceph-mon[29469]: Deploying daemon osd.3 on smithi078 2024-04-25T01:38:52.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:51 smithi078 ceph-mon[29469]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 189 KiB/s wr, 12 op/s 2024-04-25T01:38:52.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:51 smithi152 ceph-mon[32465]: Deploying daemon osd.3 on smithi078 2024-04-25T01:38:52.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:51 smithi152 ceph-mon[32465]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 189 KiB/s wr, 12 op/s 2024-04-25T01:38:52.355 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:51 smithi196 ceph-mon[32978]: Deploying daemon osd.3 on smithi078 2024-04-25T01:38:52.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:51 smithi196 ceph-mon[32978]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 189 KiB/s wr, 12 op/s 2024-04-25T01:38:54.133 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:53 smithi078 ceph-mon[29469]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-04-25T01:38:54.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:53 smithi152 ceph-mon[32465]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-04-25T01:38:54.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:53 smithi196 ceph-mon[32978]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-04-25T01:38:56.158 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:55 smithi196 ceph-mon[32978]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 160 KiB/s wr, 10 op/s 2024-04-25T01:38:56.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:55 smithi078 ceph-mon[29469]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 160 KiB/s wr, 10 op/s 2024-04-25T01:38:56.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:55 smithi152 ceph-mon[32465]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 160 KiB/s wr, 10 op/s 2024-04-25T01:38:57.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:57 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:57.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:57 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:57.496 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:57 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:57.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:57 smithi078 ceph-mon[29469]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-04-25T01:38:57.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:57 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:57.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:57 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:57.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:57 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:57.538 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:57 smithi152 ceph-mon[32465]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-04-25T01:38:57.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:57 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:57.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:57 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:38:57.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:57 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:57.606 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:57 smithi196 ceph-mon[32978]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-04-25T01:38:59.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:59.906+0000 7fe50ffff700 1 -- 172.21.15.78:0/592216339 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe528062670 con 0x7fe5100a2b90 2024-04-25T01:38:59.888 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 3 on host 'smithi078' 2024-04-25T01:38:59.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:59.908+0000 7fe52c802700 1 -- 172.21.15.78:0/592216339 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe5100a2b90 msgr2=0x7fe5100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:59.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:59.908+0000 7fe52c802700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe5100a2b90 0x7fe5100a5050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe518003890 tx=0x7fe518005fb0 comp rx=0 tx=0).stop 2024-04-25T01:38:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:59.908+0000 7fe52c802700 1 -- 172.21.15.78:0/592216339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe528103d70 msgr2=0x7fe52819f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:59.908+0000 7fe52c802700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe528103d70 0x7fe52819f9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe52000b9a0 tx=0x7fe52000bcb0 comp rx=0 tx=0).stop 2024-04-25T01:38:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:59.908+0000 7fe52c802700 1 -- 172.21.15.78:0/592216339 shutdown_connections 2024-04-25T01:38:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:59.908+0000 7fe52c802700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe5100a2b90 0x7fe5100a5050 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:59.908+0000 7fe52c802700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe528103d70 0x7fe52819f9c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:59.908+0000 7fe52c802700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5281021d0 0x7fe528198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:59.908+0000 7fe52c802700 1 --2- 172.21.15.78:0/592216339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5281033d0 0x7fe528198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:59.908+0000 7fe52c802700 1 -- 172.21.15.78:0/592216339 >> 172.21.15.78:0/592216339 conn(0x7fe5280fd770 msgr2=0x7fe5281067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:59.909+0000 7fe52c802700 1 -- 172.21.15.78:0/592216339 shutdown_connections 2024-04-25T01:38:59.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:38:59.909+0000 7fe52c802700 1 -- 172.21.15.78:0/592216339 wait complete. 2024-04-25T01:39:00.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:59 smithi196 ceph-mon[32978]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:39:00.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:59 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:00.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:59 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:00.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:59 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:00.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:59 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:00.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:38:59 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:00.153 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:59 smithi078 ceph-mon[29469]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:39:00.153 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:59 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:00.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:59 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:00.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:59 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:00.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:59 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:00.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:38:59 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:00.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:59 smithi152 ceph-mon[32465]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:39:00.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:59 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:00.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:59 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:00.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:59 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:00.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:59 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:00.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:38:59 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:00.529 DEBUG:teuthology.orchestra.run.smithi078:osd.3> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T01:39:00.531 INFO:tasks.cephadm:Deploying osd.4 on smithi152 with /dev/vg_nvme/lv_3... 2024-04-25T01:39:00.532 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T01:39:00.778 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:39:01.148 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:00 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:01.148 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:00 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:01.149 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:00 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:01.149 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:00 smithi152 ceph-mon[32465]: from='osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T01:39:01.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:00 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:01.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:00 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:01.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:00 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:01.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:00 smithi078 ceph-mon[29469]: from='osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T01:39:01.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:00 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:01.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:00 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:01.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:00 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:01.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:00 smithi196 ceph-mon[32978]: from='osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T01:39:02.175 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:01 smithi078 ceph-mon[29469]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:39:02.175 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:01 smithi078 ceph-mon[29469]: from='osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T01:39:02.175 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:01 smithi078 ceph-mon[29469]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T01:39:02.175 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:01 smithi078 ceph-mon[29469]: from='osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T01:39:02.175 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:01 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:39:02.175 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:39:01 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3[48445]: 2024-04-25T01:39:01.915+0000 7fed290a9700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T01:39:02.175 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:39:01 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3[48445]: 2024-04-25T01:39:01.921+0000 7fed24e9c700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:39:02.180 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:01 smithi152 ceph-mon[32465]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:39:02.180 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:01 smithi152 ceph-mon[32465]: from='osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T01:39:02.180 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:01 smithi152 ceph-mon[32465]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T01:39:02.181 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:01 smithi152 ceph-mon[32465]: from='osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T01:39:02.181 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:01 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:39:02.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:01 smithi196 ceph-mon[32978]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:39:02.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:01 smithi196 ceph-mon[32978]: from='osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T01:39:02.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:01 smithi196 ceph-mon[32978]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T01:39:02.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:01 smithi196 ceph-mon[32978]: from='osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T01:39:02.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:01 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:39:02.730 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T01:39:02.730 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T01:39:02.730 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-04-25T01:39:02.730 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-04-25T01:39:02.730 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114819 s, 913 MB/s 2024-04-25T01:39:02.730 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-04-25T01:39:03.128 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:02 smithi078 ceph-mon[29469]: from='osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T01:39:03.128 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:02 smithi078 ceph-mon[29469]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T01:39:03.128 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:39:03.128 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:39:03.129 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:02 smithi078 ceph-mon[29469]: osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265] boot 2024-04-25T01:39:03.129 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:02 smithi078 ceph-mon[29469]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T01:39:03.129 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:39:03.283 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:02 smithi152 ceph-mon[32465]: from='osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T01:39:03.283 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:02 smithi152 ceph-mon[32465]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T01:39:03.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:39:03.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:39:03.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:02 smithi152 ceph-mon[32465]: osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265] boot 2024-04-25T01:39:03.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:02 smithi152 ceph-mon[32465]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T01:39:03.284 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:39:03.311 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi152:vg_nvme/lv_3 2024-04-25T01:39:03.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:02 smithi196 ceph-mon[32978]: from='osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T01:39:03.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:02 smithi196 ceph-mon[32978]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T01:39:03.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:39:03.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:39:03.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:02 smithi196 ceph-mon[32978]: osd.3 [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265] boot 2024-04-25T01:39:03.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:02 smithi196 ceph-mon[32978]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T01:39:03.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:39:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:39:04.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:03 smithi078 ceph-mon[29469]: purged_snaps scrub starts 2024-04-25T01:39:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:03 smithi078 ceph-mon[29469]: purged_snaps scrub ok 2024-04-25T01:39:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:03 smithi078 ceph-mon[29469]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:39:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:03 smithi078 ceph-mon[29469]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T01:39:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:04.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:04.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:04.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:03 smithi152 ceph-mon[32465]: purged_snaps scrub starts 2024-04-25T01:39:04.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:03 smithi152 ceph-mon[32465]: purged_snaps scrub ok 2024-04-25T01:39:04.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:03 smithi152 ceph-mon[32465]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:39:04.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:03 smithi152 ceph-mon[32465]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T01:39:04.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:04.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:04.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:04.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:04.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:04.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:04.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:04.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:04.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:03 smithi196 ceph-mon[32978]: purged_snaps scrub starts 2024-04-25T01:39:04.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:03 smithi196 ceph-mon[32978]: purged_snaps scrub ok 2024-04-25T01:39:04.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:03 smithi196 ceph-mon[32978]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T01:39:04.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:03 smithi196 ceph-mon[32978]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T01:39:04.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:04.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:04.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:04.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:04.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:04.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:04.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:04.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:04.901 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:04 smithi152 ceph-mon[32465]: Detected new or changed devices on smithi078 2024-04-25T01:39:05.167 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:04 smithi152 ceph-mon[32465]: Adjusting osd_memory_target on smithi078 to 8544M 2024-04-25T01:39:05.167 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:04 smithi152 ceph-mon[32465]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T01:39:05.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.198+0000 7f6cd89aa700 1 -- 172.21.15.152:0/1392271871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cd41005a0 msgr2=0x7f6cd41009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:05.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.198+0000 7f6cd89aa700 1 --2- 172.21.15.152:0/1392271871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cd41005a0 0x7f6cd41009c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009990 tx=0x7f6cc8009ca0 comp rx=0 tx=0).stop 2024-04-25T01:39:05.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.198+0000 7f6cd89aa700 1 -- 172.21.15.152:0/1392271871 shutdown_connections 2024-04-25T01:39:05.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.198+0000 7f6cd89aa700 1 --2- 172.21.15.152:0/1392271871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cd4103f20 0x7f6cd410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:05.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.198+0000 7f6cd89aa700 1 --2- 172.21.15.152:0/1392271871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cd41005a0 0x7f6cd41009c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:05.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.198+0000 7f6cd89aa700 1 --2- 172.21.15.152:0/1392271871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd4103580 0x7f6cd41039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:05.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.198+0000 7f6cd89aa700 1 -- 172.21.15.152:0/1392271871 >> 172.21.15.152:0/1392271871 conn(0x7f6cd40fbf00 msgr2=0x7f6cd40fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:05.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.198+0000 7f6cd89aa700 1 -- 172.21.15.152:0/1392271871 shutdown_connections 2024-04-25T01:39:05.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.198+0000 7f6cd89aa700 1 -- 172.21.15.152:0/1392271871 wait complete. 2024-04-25T01:39:05.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.199+0000 7f6cd89aa700 1 Processor -- start 2024-04-25T01:39:05.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.199+0000 7f6cd89aa700 1 -- start start 2024-04-25T01:39:05.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.199+0000 7f6cd89aa700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd41005a0 0x7f6cd4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:05.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.199+0000 7f6cd89aa700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cd4103580 0x7f6cd41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:05.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.199+0000 7f6cd89aa700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cd4103f20 0x7f6cd419fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:05.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.199+0000 7f6cd89aa700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd410deb0 con 0x7f6cd4103f20 2024-04-25T01:39:05.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.199+0000 7f6cd89aa700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd410dd30 con 0x7f6cd4103580 2024-04-25T01:39:05.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.199+0000 7f6cd89aa700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd410e030 con 0x7f6cd41005a0 2024-04-25T01:39:05.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cd1d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cd4103580 0x7f6cd41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:05.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cd1d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cd4103580 0x7f6cd41989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57240/0 (socket says 172.21.15.152:57240) 2024-04-25T01:39:05.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cd1d9b700 1 -- 172.21.15.152:0/2392261386 learned_addr learned my addr 172.21.15.152:0/2392261386 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-25T01:39:05.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cd259c700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd41005a0 0x7f6cd4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:05.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cd2d9d700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cd4103f20 0x7f6cd419fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:05.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cd1d9b700 1 -- 172.21.15.152:0/2392261386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd41005a0 msgr2=0x7f6cd4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:05.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cd1d9b700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd41005a0 0x7f6cd4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:05.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cd1d9b700 1 -- 172.21.15.152:0/2392261386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cd4103f20 msgr2=0x7f6cd419fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:05.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cd1d9b700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cd4103f20 0x7f6cd419fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:05.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cd1d9b700 1 -- 172.21.15.152:0/2392261386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd41a00d0 con 0x7f6cd4103580 2024-04-25T01:39:05.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cd259c700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd41005a0 0x7f6cd4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:39:05.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cd1d9b700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cd4103580 0x7f6cd41989b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400c970 tx=0x7f6cc400cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:05.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cc37fe700 1 -- 172.21.15.152:0/2392261386 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4007aa0 con 0x7f6cd4103580 2024-04-25T01:39:05.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cc37fe700 1 -- 172.21.15.152:0/2392261386 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc4004d10 con 0x7f6cd4103580 2024-04-25T01:39:05.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cd89aa700 1 -- 172.21.15.152:0/2392261386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd41a03c0 con 0x7f6cd4103580 2024-04-25T01:39:05.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cc37fe700 1 -- 172.21.15.152:0/2392261386 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400f450 con 0x7f6cd4103580 2024-04-25T01:39:05.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.200+0000 7f6cd89aa700 1 -- 172.21.15.152:0/2392261386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd41a0900 con 0x7f6cd4103580 2024-04-25T01:39:05.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.201+0000 7f6cd89aa700 1 -- 172.21.15.152:0/2392261386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd4068f20 con 0x7f6cd4103580 2024-04-25T01:39:05.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.202+0000 7f6cc37fe700 1 -- 172.21.15.152:0/2392261386 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6cc4004750 con 0x7f6cd4103580 2024-04-25T01:39:05.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.203+0000 7f6cc37fe700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6cbc0a2ca0 0x7f6cbc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:05.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.203+0000 7f6cc37fe700 1 -- 172.21.15.152:0/2392261386 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f6cc40cde80 con 0x7f6cd4103580 2024-04-25T01:39:05.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.203+0000 7f6cd259c700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6cbc0a2ca0 0x7f6cbc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:05.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.203+0000 7f6cd259c700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6cbc0a2ca0 0x7f6cbc0a5160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80039c0 tx=0x7f6cc8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:05.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.206+0000 7f6cc37fe700 1 -- 172.21.15.152:0/2392261386 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6cc400de20 con 0x7f6cd4103580 2024-04-25T01:39:05.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:04 smithi078 ceph-mon[29469]: Detected new or changed devices on smithi078 2024-04-25T01:39:05.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:04 smithi078 ceph-mon[29469]: Adjusting osd_memory_target on smithi078 to 8544M 2024-04-25T01:39:05.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:04 smithi078 ceph-mon[29469]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T01:39:05.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:04 smithi196 ceph-mon[32978]: Detected new or changed devices on smithi078 2024-04-25T01:39:05.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:04 smithi196 ceph-mon[32978]: Adjusting osd_memory_target on smithi078 to 8544M 2024-04-25T01:39:05.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:04 smithi196 ceph-mon[32978]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T01:39:05.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:05.413+0000 7f6cd89aa700 1 -- 172.21.15.152:0/2392261386 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6cd4062670 con 0x7f6cbc0a2ca0 2024-04-25T01:39:06.245 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:05 smithi152 ceph-mon[32465]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:06.246 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:05 smithi152 ceph-mon[32465]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T01:39:06.246 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:05 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:39:06.246 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:05 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:39:06.246 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:05 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:06.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:05 smithi078 ceph-mon[29469]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:06.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:05 smithi078 ceph-mon[29469]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T01:39:06.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:39:06.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:39:06.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:06.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:05 smithi196 ceph-mon[32978]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:06.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:05 smithi196 ceph-mon[32978]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T01:39:06.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:39:06.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:39:06.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:07.161 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:06 smithi078 ceph-mon[29469]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:07.161 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:06 smithi078 ceph-mon[29469]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T01:39:07.169 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:06 smithi152 ceph-mon[32465]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:07.170 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:06 smithi152 ceph-mon[32465]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T01:39:07.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:06 smithi196 ceph-mon[32978]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:07.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:06 smithi196 ceph-mon[32978]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T01:39:08.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:07 smithi078 ceph-mon[29469]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T01:39:08.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:07 smithi152 ceph-mon[32465]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T01:39:08.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:07 smithi196 ceph-mon[32978]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T01:39:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:08 smithi078 ceph-mon[29469]: from='client.? 172.21.15.152:0/570528232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6def52d6-f070-42a6-80a9-318b18cfe4a9"}]: dispatch 2024-04-25T01:39:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:08 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6def52d6-f070-42a6-80a9-318b18cfe4a9"}]: dispatch 2024-04-25T01:39:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:08 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6def52d6-f070-42a6-80a9-318b18cfe4a9"}]': finished 2024-04-25T01:39:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:08 smithi078 ceph-mon[29469]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T01:39:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:08 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:09.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:08 smithi078 ceph-mon[29469]: from='client.? 172.21.15.152:0/621527208' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:39:09.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:08 smithi152 ceph-mon[32465]: from='client.? 172.21.15.152:0/570528232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6def52d6-f070-42a6-80a9-318b18cfe4a9"}]: dispatch 2024-04-25T01:39:09.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:08 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6def52d6-f070-42a6-80a9-318b18cfe4a9"}]: dispatch 2024-04-25T01:39:09.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:08 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6def52d6-f070-42a6-80a9-318b18cfe4a9"}]': finished 2024-04-25T01:39:09.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:08 smithi152 ceph-mon[32465]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T01:39:09.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:08 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:09.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:08 smithi152 ceph-mon[32465]: from='client.? 172.21.15.152:0/621527208' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:39:09.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:08 smithi196 ceph-mon[32978]: from='client.? 172.21.15.152:0/570528232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6def52d6-f070-42a6-80a9-318b18cfe4a9"}]: dispatch 2024-04-25T01:39:09.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:08 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6def52d6-f070-42a6-80a9-318b18cfe4a9"}]: dispatch 2024-04-25T01:39:09.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:08 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6def52d6-f070-42a6-80a9-318b18cfe4a9"}]': finished 2024-04-25T01:39:09.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:08 smithi196 ceph-mon[32978]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T01:39:09.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:08 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:09.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:08 smithi196 ceph-mon[32978]: from='client.? 172.21.15.152:0/621527208' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:39:10.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:09 smithi078 ceph-mon[29469]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-25T01:39:10.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:09 smithi152 ceph-mon[32465]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-25T01:39:10.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:09 smithi196 ceph-mon[32978]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-25T01:39:12.039 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:11 smithi152 ceph-mon[32465]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T01:39:12.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:11 smithi078 ceph-mon[29469]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T01:39:12.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:11 smithi196 ceph-mon[32978]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T01:39:14.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:13 smithi078 ceph-mon[29469]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-25T01:39:14.272 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:13 smithi152 ceph-mon[32465]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-25T01:39:14.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:13 smithi196 ceph-mon[32978]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-25T01:39:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:15 smithi078 ceph-mon[29469]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T01:39:16.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:16.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:15 smithi152 ceph-mon[32465]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:16.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T01:39:16.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:16.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:15 smithi196 ceph-mon[32978]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:16.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T01:39:16.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:17.210 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:16 smithi152 ceph-mon[32465]: Deploying daemon osd.4 on smithi152 2024-04-25T01:39:17.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:16 smithi078 ceph-mon[29469]: Deploying daemon osd.4 on smithi152 2024-04-25T01:39:17.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:16 smithi196 ceph-mon[32978]: Deploying daemon osd.4 on smithi152 2024-04-25T01:39:18.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:17 smithi078 ceph-mon[29469]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:18.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:17 smithi152 ceph-mon[32465]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:18.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:17 smithi196 ceph-mon[32978]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:20.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:19 smithi078 ceph-mon[29469]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:20.288 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:19 smithi152 ceph-mon[32465]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:20.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:19 smithi196 ceph-mon[32978]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:22.110 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:21 smithi152 ceph-mon[32465]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:22.110 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:21 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:22.110 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:21 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:22.111 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:21 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:22.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:21 smithi078 ceph-mon[29469]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:22.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:21 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:22.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:21 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:22.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:21 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:22.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:21 smithi196 ceph-mon[32978]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:22.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:21 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:22.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:21 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:22.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:21 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:24.039 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:23 smithi152 ceph-mon[32465]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:24.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:23 smithi078 ceph-mon[29469]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:24.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:23 smithi196 ceph-mon[32978]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:25.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:25.027+0000 7f6cc37fe700 1 -- 172.21.15.152:0/2392261386 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6cd4062670 con 0x7f6cbc0a2ca0 2024-04-25T01:39:25.010 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 4 on host 'smithi152' 2024-04-25T01:39:25.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:25.030+0000 7f6cd89aa700 1 -- 172.21.15.152:0/2392261386 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6cbc0a2ca0 msgr2=0x7f6cbc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:25.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:25.030+0000 7f6cd89aa700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6cbc0a2ca0 0x7f6cbc0a5160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80039c0 tx=0x7f6cc8005fb0 comp rx=0 tx=0).stop 2024-04-25T01:39:25.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:25.030+0000 7f6cd89aa700 1 -- 172.21.15.152:0/2392261386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cd4103580 msgr2=0x7f6cd41989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:25.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:25.030+0000 7f6cd89aa700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cd4103580 0x7f6cd41989b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400c970 tx=0x7f6cc400cd30 comp rx=0 tx=0).stop 2024-04-25T01:39:25.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:25.030+0000 7f6cd89aa700 1 -- 172.21.15.152:0/2392261386 shutdown_connections 2024-04-25T01:39:25.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:25.030+0000 7f6cd89aa700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6cbc0a2ca0 0x7f6cbc0a5160 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:25.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:25.030+0000 7f6cd89aa700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cd41005a0 0x7f6cd4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:25.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:25.030+0000 7f6cd89aa700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6cd4103f20 0x7f6cd419fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:25.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:25.030+0000 7f6cd89aa700 1 --2- 172.21.15.152:0/2392261386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6cd4103580 0x7f6cd41989b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:25.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:25.030+0000 7f6cd89aa700 1 -- 172.21.15.152:0/2392261386 >> 172.21.15.152:0/2392261386 conn(0x7f6cd40fbf00 msgr2=0x7f6cd4106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:25.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:25.030+0000 7f6cd89aa700 1 -- 172.21.15.152:0/2392261386 shutdown_connections 2024-04-25T01:39:25.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:39:25.030+0000 7f6cd89aa700 1 -- 172.21.15.152:0/2392261386 wait complete. 2024-04-25T01:39:25.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:25.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:25.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:25 smithi152 ceph-mon[32465]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:25.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:25.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:25.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:25.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:25 smithi196 ceph-mon[32978]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:25.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:25.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:25.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:25.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:25 smithi078 ceph-mon[29469]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:25.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:25.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:25.659 DEBUG:teuthology.orchestra.run.smithi152:osd.4> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T01:39:25.661 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/vg_nvme/lv_3... 2024-04-25T01:39:25.661 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T01:39:25.912 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:39:26.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:26 smithi196 ceph-mon[32978]: from='osd.4 [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:39:26.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:26 smithi196 ceph-mon[32978]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:39:26.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:26 smithi152 ceph-mon[32465]: from='osd.4 [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:39:26.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:26 smithi152 ceph-mon[32465]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:39:26.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:26 smithi078 ceph-mon[29469]: from='osd.4 [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:39:26.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:26 smithi078 ceph-mon[29469]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:39:27.153 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:27 smithi152 ceph-mon[32465]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T01:39:27.153 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:27 smithi152 ceph-mon[32465]: from='osd.4 [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:39:27.153 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:27 smithi152 ceph-mon[32465]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T01:39:27.153 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:27 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:27.153 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:27 smithi152 ceph-mon[32465]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:39:27.153 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:27 smithi152 ceph-mon[32465]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:27.200 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:27 smithi196 ceph-mon[32978]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T01:39:27.200 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:27 smithi196 ceph-mon[32978]: from='osd.4 [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:39:27.201 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:27 smithi196 ceph-mon[32978]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T01:39:27.201 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:27 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:27.201 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:27 smithi196 ceph-mon[32978]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:39:27.201 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:27 smithi196 ceph-mon[32978]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:27.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:27 smithi078 ceph-mon[29469]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T01:39:27.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:27 smithi078 ceph-mon[29469]: from='osd.4 [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:39:27.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:27 smithi078 ceph-mon[29469]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T01:39:27.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:27 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:27.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:27 smithi078 ceph-mon[29469]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:39:27.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:27 smithi078 ceph-mon[29469]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:27.772 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T01:39:27.773 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T01:39:27.773 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-04-25T01:39:27.773 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-04-25T01:39:27.773 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130248 s, 805 MB/s 2024-04-25T01:39:27.773 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-04-25T01:39:27.789 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:39:27 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4[43979]: 2024-04-25T01:39:27.551+0000 7f718789b700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T01:39:27.789 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:39:27 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4[43979]: 2024-04-25T01:39:27.558+0000 7f717f686700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:39:28.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:28 smithi152 ceph-mon[32465]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T01:39:28.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:28 smithi152 ceph-mon[32465]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T01:39:28.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:28 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:28.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:28 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:28.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:28 smithi152 ceph-mon[32465]: from='osd.4 ' entity='osd.4' 2024-04-25T01:39:28.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:28 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:28.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:28 smithi196 ceph-mon[32978]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T01:39:28.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:28 smithi196 ceph-mon[32978]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T01:39:28.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:28 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:28.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:28 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:28.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:28 smithi196 ceph-mon[32978]: from='osd.4 ' entity='osd.4' 2024-04-25T01:39:28.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:28 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:28.411 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-04-25T01:39:28.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:28 smithi078 ceph-mon[29469]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T01:39:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:28 smithi078 ceph-mon[29469]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T01:39:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:28 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:28 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:28 smithi078 ceph-mon[29469]: from='osd.4 ' entity='osd.4' 2024-04-25T01:39:28.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:28 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:28.653 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:39:29.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: purged_snaps scrub starts 2024-04-25T01:39:29.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: purged_snaps scrub ok 2024-04-25T01:39:29.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: osd.4 [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312] boot 2024-04-25T01:39:29.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T01:39:29.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:29.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: Detected new or changed devices on smithi152 2024-04-25T01:39:29.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:29.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:29.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:29.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:29.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: Adjusting osd_memory_target on smithi152 to 8544M 2024-04-25T01:39:29.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:29.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:29.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:29.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:29.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:29 smithi196 ceph-mon[32978]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:29.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: purged_snaps scrub starts 2024-04-25T01:39:29.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: purged_snaps scrub ok 2024-04-25T01:39:29.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: osd.4 [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312] boot 2024-04-25T01:39:29.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T01:39:29.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:29.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: Detected new or changed devices on smithi152 2024-04-25T01:39:29.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:29.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:29.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:29.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:29.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: Adjusting osd_memory_target on smithi152 to 8544M 2024-04-25T01:39:29.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:29.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:29.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:29.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:29.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:29 smithi078 ceph-mon[29469]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:29.539 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: purged_snaps scrub starts 2024-04-25T01:39:29.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: purged_snaps scrub ok 2024-04-25T01:39:29.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: osd.4 [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312] boot 2024-04-25T01:39:29.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T01:39:29.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:39:29.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: Detected new or changed devices on smithi152 2024-04-25T01:39:29.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:29.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:29.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:29.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:29.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: Adjusting osd_memory_target on smithi152 to 8544M 2024-04-25T01:39:29.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:29.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:29.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:29.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:29.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:29 smithi152 ceph-mon[32465]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T01:39:30.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.253+0000 7f53285e4700 1 -- 172.21.15.196:0/2058969457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5320102dc0 msgr2=0x7f53201051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:30.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.253+0000 7f53285e4700 1 --2- 172.21.15.196:0/2058969457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5320102dc0 0x7f53201051b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5310009990 tx=0x7f5310009ca0 comp rx=0 tx=0).stop 2024-04-25T01:39:30.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.253+0000 7f53285e4700 1 -- 172.21.15.196:0/2058969457 shutdown_connections 2024-04-25T01:39:30.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.253+0000 7f53285e4700 1 --2- 172.21.15.196:0/2058969457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5320102dc0 0x7f53201051b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:30.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.253+0000 7f53285e4700 1 --2- 172.21.15.196:0/2058969457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5320108020 0x7f532010a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:30.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.253+0000 7f53285e4700 1 --2- 172.21.15.196:0/2058969457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53201056f0 0x7f5320107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:30.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.253+0000 7f53285e4700 1 -- 172.21.15.196:0/2058969457 >> 172.21.15.196:0/2058969457 conn(0x7f53200fc790 msgr2=0x7f53200febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:30.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.253+0000 7f53285e4700 1 -- 172.21.15.196:0/2058969457 shutdown_connections 2024-04-25T01:39:30.238 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.253+0000 7f53285e4700 1 -- 172.21.15.196:0/2058969457 wait complete. 2024-04-25T01:39:30.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.254+0000 7f53285e4700 1 Processor -- start 2024-04-25T01:39:30.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.254+0000 7f53285e4700 1 -- start start 2024-04-25T01:39:30.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.254+0000 7f53285e4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5320102dc0 0x7f53201984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:30.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.254+0000 7f53285e4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53201056f0 0x7f53201989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:30.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.254+0000 7f53285e4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5320108020 0x7f532019fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:30.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.254+0000 7f53285e4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f532010df20 con 0x7f5320102dc0 2024-04-25T01:39:30.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.254+0000 7f53285e4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f532010dda0 con 0x7f5320108020 2024-04-25T01:39:30.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.254+0000 7f53285e4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f532010e0a0 con 0x7f53201056f0 2024-04-25T01:39:30.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.254+0000 7f5325b7f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53201056f0 0x7f53201989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:30.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.254+0000 7f5325b7f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53201056f0 0x7f53201989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:40488/0 (socket says 172.21.15.196:40488) 2024-04-25T01:39:30.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.254+0000 7f5325b7f700 1 -- 172.21.15.196:0/3462039532 learned_addr learned my addr 172.21.15.196:0/3462039532 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T01:39:30.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.254+0000 7f5326380700 1 --2- 172.21.15.196:0/3462039532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5320102dc0 0x7f53201984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:30.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.255+0000 7f5325b7f700 1 -- 172.21.15.196:0/3462039532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5320108020 msgr2=0x7f532019fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:30.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.255+0000 7f5325b7f700 1 --2- 172.21.15.196:0/3462039532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5320108020 0x7f532019fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:30.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.255+0000 7f5325b7f700 1 -- 172.21.15.196:0/3462039532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5320102dc0 msgr2=0x7f53201984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:30.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.255+0000 7f5325b7f700 1 --2- 172.21.15.196:0/3462039532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5320102dc0 0x7f53201984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:30.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.255+0000 7f5325b7f700 1 -- 172.21.15.196:0/3462039532 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53201a0100 con 0x7f53201056f0 2024-04-25T01:39:30.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.255+0000 7f5325b7f700 1 --2- 172.21.15.196:0/3462039532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53201056f0 0x7f53201989e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f531c00d980 tx=0x7f531c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:30.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.255+0000 7f53177fe700 1 -- 172.21.15.196:0/3462039532 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c016070 con 0x7f53201056f0 2024-04-25T01:39:30.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.255+0000 7f53177fe700 1 -- 172.21.15.196:0/3462039532 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f531c00de70 con 0x7f53201056f0 2024-04-25T01:39:30.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.255+0000 7f53285e4700 1 -- 172.21.15.196:0/3462039532 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53201a03f0 con 0x7f53201056f0 2024-04-25T01:39:30.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.255+0000 7f53177fe700 1 -- 172.21.15.196:0/3462039532 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c00f3c0 con 0x7f53201056f0 2024-04-25T01:39:30.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.255+0000 7f53285e4700 1 -- 172.21.15.196:0/3462039532 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53201a0a00 con 0x7f53201056f0 2024-04-25T01:39:30.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.256+0000 7f53285e4700 1 -- 172.21.15.196:0/3462039532 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5320068f20 con 0x7f53201056f0 2024-04-25T01:39:30.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.257+0000 7f53177fe700 1 -- 172.21.15.196:0/3462039532 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f531c019070 con 0x7f53201056f0 2024-04-25T01:39:30.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.257+0000 7f53177fe700 1 --2- 172.21.15.196:0/3462039532 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f530c0a2ca0 0x7f530c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:30.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.258+0000 7f53177fe700 1 -- 172.21.15.196:0/3462039532 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f531c0cda90 con 0x7f53201056f0 2024-04-25T01:39:30.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.258+0000 7f5326380700 1 --2- 172.21.15.196:0/3462039532 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f530c0a2ca0 0x7f530c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:30.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.258+0000 7f5326380700 1 --2- 172.21.15.196:0/3462039532 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f530c0a2ca0 0x7f530c0a5160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f531000b5c0 tx=0x7f5310005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:30.244 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.262+0000 7f53177fe700 1 -- 172.21.15.196:0/3462039532 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f531c09ba60 con 0x7f53201056f0 2024-04-25T01:39:30.448 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:30.466+0000 7f53285e4700 1 -- 172.21.15.196:0/3462039532 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5320062670 con 0x7f530c0a2ca0 2024-04-25T01:39:31.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:31 smithi196 ceph-mon[32978]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T01:39:31.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:31 smithi196 ceph-mon[32978]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:31.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:31 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:39:31.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:31 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:39:31.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:31 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:31.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:31 smithi196 ceph-mon[32978]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:31.497 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:31 smithi078 ceph-mon[29469]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T01:39:31.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:31 smithi078 ceph-mon[29469]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:31.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:31 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:39:31.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:31 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:39:31.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:31 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:31.498 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:31 smithi078 ceph-mon[29469]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:31.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:31 smithi152 ceph-mon[32465]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T01:39:31.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:31 smithi152 ceph-mon[32465]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:31.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:31 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:39:31.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:31 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:39:31.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:31 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:31.540 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:31 smithi152 ceph-mon[32465]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:34.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:33 smithi078 ceph-mon[29469]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:34.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:33 smithi078 ceph-mon[29469]: from='client.? 172.21.15.196:0/3213402848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "537029e6-890b-47da-973c-c3397db9751e"}]: dispatch 2024-04-25T01:39:34.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:33 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "537029e6-890b-47da-973c-c3397db9751e"}]: dispatch 2024-04-25T01:39:34.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:33 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "537029e6-890b-47da-973c-c3397db9751e"}]': finished 2024-04-25T01:39:34.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:33 smithi078 ceph-mon[29469]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T01:39:34.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:33 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:34.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:33 smithi078 ceph-mon[29469]: from='client.? 172.21.15.196:0/3039715121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:39:34.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:33 smithi152 ceph-mon[32465]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:34.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:33 smithi152 ceph-mon[32465]: from='client.? 172.21.15.196:0/3213402848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "537029e6-890b-47da-973c-c3397db9751e"}]: dispatch 2024-04-25T01:39:34.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:33 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "537029e6-890b-47da-973c-c3397db9751e"}]: dispatch 2024-04-25T01:39:34.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:33 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "537029e6-890b-47da-973c-c3397db9751e"}]': finished 2024-04-25T01:39:34.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:33 smithi152 ceph-mon[32465]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T01:39:34.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:33 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:34.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:33 smithi152 ceph-mon[32465]: from='client.? 172.21.15.196:0/3039715121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:39:34.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:33 smithi196 ceph-mon[32978]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:34.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:33 smithi196 ceph-mon[32978]: from='client.? 172.21.15.196:0/3213402848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "537029e6-890b-47da-973c-c3397db9751e"}]: dispatch 2024-04-25T01:39:34.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:33 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "537029e6-890b-47da-973c-c3397db9751e"}]: dispatch 2024-04-25T01:39:34.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:33 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "537029e6-890b-47da-973c-c3397db9751e"}]': finished 2024-04-25T01:39:34.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:33 smithi196 ceph-mon[32978]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T01:39:34.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:33 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:34.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:33 smithi196 ceph-mon[32978]: from='client.? 172.21.15.196:0/3039715121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:39:36.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:35 smithi078 ceph-mon[29469]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:36.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:35 smithi152 ceph-mon[32465]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:36.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:35 smithi196 ceph-mon[32978]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:38.139 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:37 smithi196 ceph-mon[32978]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:38.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:37 smithi078 ceph-mon[29469]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:38.289 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:37 smithi152 ceph-mon[32465]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:40.110 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:39 smithi196 ceph-mon[32978]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:40.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:39 smithi078 ceph-mon[29469]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:40.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:39 smithi152 ceph-mon[32465]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:41.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:40 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T01:39:41.188 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:40 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:41.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:40 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T01:39:41.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:40 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:41.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:40 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T01:39:41.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:40 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:42.191 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:41 smithi196 ceph-mon[32978]: Deploying daemon osd.5 on smithi196 2024-04-25T01:39:42.191 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:41 smithi196 ceph-mon[32978]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:42.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:41 smithi078 ceph-mon[29469]: Deploying daemon osd.5 on smithi196 2024-04-25T01:39:42.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:41 smithi078 ceph-mon[29469]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:42.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:41 smithi152 ceph-mon[32465]: Deploying daemon osd.5 on smithi196 2024-04-25T01:39:42.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:41 smithi152 ceph-mon[32465]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:44.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:43 smithi078 ceph-mon[29469]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:44.268 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:43 smithi196 ceph-mon[32978]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:44.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:43 smithi152 ceph-mon[32465]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:46.021 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:45 smithi196 ceph-mon[32978]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:46.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:45 smithi078 ceph-mon[29469]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:46.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:45 smithi152 ceph-mon[32465]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:47.608 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:47 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:47.608 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:47 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:47.608 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:47 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:47.608 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:47 smithi196 ceph-mon[32978]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:47.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:47.748 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:47.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:47.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:47 smithi078 ceph-mon[29469]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:47.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:47 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:47.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:47 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:47.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:47 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:47.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:47 smithi152 ceph-mon[32465]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:49.974 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:49.990+0000 7f53177fe700 1 -- 172.21.15.196:0/3462039532 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5320062670 con 0x7f530c0a2ca0 2024-04-25T01:39:49.974 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-04-25T01:39:49.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:49.992+0000 7f53285e4700 1 -- 172.21.15.196:0/3462039532 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f530c0a2ca0 msgr2=0x7f530c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:49.976 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:49.992+0000 7f53285e4700 1 --2- 172.21.15.196:0/3462039532 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f530c0a2ca0 0x7f530c0a5160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f531000b5c0 tx=0x7f5310005fd0 comp rx=0 tx=0).stop 2024-04-25T01:39:49.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:49.992+0000 7f53285e4700 1 -- 172.21.15.196:0/3462039532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53201056f0 msgr2=0x7f53201989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:49.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:49.992+0000 7f53285e4700 1 --2- 172.21.15.196:0/3462039532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53201056f0 0x7f53201989e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f531c00d980 tx=0x7f531c00dc90 comp rx=0 tx=0).stop 2024-04-25T01:39:49.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:49.992+0000 7f53285e4700 1 -- 172.21.15.196:0/3462039532 shutdown_connections 2024-04-25T01:39:49.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:49.992+0000 7f53285e4700 1 --2- 172.21.15.196:0/3462039532 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f530c0a2ca0 0x7f530c0a5160 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:49.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:49.992+0000 7f53285e4700 1 --2- 172.21.15.196:0/3462039532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5320108020 0x7f532019fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:49.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:49.992+0000 7f53285e4700 1 --2- 172.21.15.196:0/3462039532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5320102dc0 0x7f53201984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:49.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:49.992+0000 7f53285e4700 1 --2- 172.21.15.196:0/3462039532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53201056f0 0x7f53201989e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:49.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:49.992+0000 7f53285e4700 1 -- 172.21.15.196:0/3462039532 >> 172.21.15.196:0/3462039532 conn(0x7f53200fc790 msgr2=0x7f5320103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:49.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:49.992+0000 7f53285e4700 1 -- 172.21.15.196:0/3462039532 shutdown_connections 2024-04-25T01:39:49.977 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:39:49.992+0000 7f53285e4700 1 -- 172.21.15.196:0/3462039532 wait complete. 2024-04-25T01:39:50.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:49 smithi078 ceph-mon[29469]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:50.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:49 smithi196 ceph-mon[32978]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:50.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:49 smithi152 ceph-mon[32465]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:50.622 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T01:39:50.625 INFO:tasks.cephadm:Deploying osd.6 on smithi078 with /dev/vg_nvme/lv_2... 2024-04-25T01:39:50.625 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T01:39:50.868 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:39:51.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:50 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:51.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:50 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:51.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:50 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:51.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:50 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:51.106 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:50 smithi196 ceph-mon[32978]: from='osd.5 [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:39:51.107 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:50 smithi196 ceph-mon[32978]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:39:51.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:50 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:51.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:50 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:51.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:50 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:51.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:50 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:51.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:50 smithi078 ceph-mon[29469]: from='osd.5 [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:39:51.208 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:50 smithi078 ceph-mon[29469]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:39:51.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:50 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:51.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:50 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:51.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:50 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:51.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:50 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:51.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:50 smithi152 ceph-mon[32465]: from='osd.5 [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:39:51.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:50 smithi152 ceph-mon[32465]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:39:52.108 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:52 smithi196 ceph-mon[32978]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:52.108 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:52 smithi196 ceph-mon[32978]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T01:39:52.108 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:52 smithi196 ceph-mon[32978]: from='osd.5 [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:39:52.108 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:52 smithi196 ceph-mon[32978]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T01:39:52.109 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:52 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:52.109 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:52 smithi196 ceph-mon[32978]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:39:52.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:52 smithi152 ceph-mon[32465]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:52.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:52 smithi152 ceph-mon[32465]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T01:39:52.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:52 smithi152 ceph-mon[32465]: from='osd.5 [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:39:52.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:52 smithi152 ceph-mon[32465]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T01:39:52.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:52 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:52.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:52 smithi152 ceph-mon[32465]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:39:52.337 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:52 smithi078 ceph-mon[29469]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:52.337 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:52 smithi078 ceph-mon[29469]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T01:39:52.337 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:52 smithi078 ceph-mon[29469]: from='osd.5 [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:39:52.337 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:52 smithi078 ceph-mon[29469]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T01:39:52.337 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:52 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:52.337 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:52 smithi078 ceph-mon[29469]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:39:52.859 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:39:52 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:39:52.441+0000 7f61ee85a700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T01:39:52.859 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:39:52 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:39:52.448+0000 7f61e6645700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:39:52.912 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T01:39:52.912 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T01:39:52.912 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-25T01:39:52.912 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-25T01:39:52.912 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115251 s, 910 MB/s 2024-04-25T01:39:52.913 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-25T01:39:53.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:53 smithi196 ceph-mon[32978]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-25T01:39:53.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:53 smithi196 ceph-mon[32978]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T01:39:53.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:53.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:53.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:53 smithi196 ceph-mon[32978]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T01:39:53.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:53.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:53 smithi196 ceph-mon[32978]: from='osd.5 ' entity='osd.5' 2024-04-25T01:39:53.269 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:53 smithi196 ceph-mon[32978]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:53.279 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:53 smithi078 ceph-mon[29469]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-25T01:39:53.279 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:53 smithi078 ceph-mon[29469]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T01:39:53.279 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:53.279 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:53.279 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:53 smithi078 ceph-mon[29469]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T01:39:53.280 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:53.280 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:53 smithi078 ceph-mon[29469]: from='osd.5 ' entity='osd.5' 2024-04-25T01:39:53.280 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:53 smithi078 ceph-mon[29469]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:53.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:53 smithi152 ceph-mon[32465]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-25T01:39:53.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:53 smithi152 ceph-mon[32465]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T01:39:53.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:53.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:53.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:53 smithi152 ceph-mon[32465]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T01:39:53.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:53.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:53 smithi152 ceph-mon[32465]: from='osd.5 ' entity='osd.5' 2024-04-25T01:39:53.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:53 smithi152 ceph-mon[32465]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-25T01:39:53.599 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_2 2024-04-25T01:39:53.849 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:39:54.217 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: purged_snaps scrub starts 2024-04-25T01:39:54.217 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: purged_snaps scrub ok 2024-04-25T01:39:54.217 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:54.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: osd.5 [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253] boot 2024-04-25T01:39:54.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T01:39:54.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:54.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: Detected new or changed devices on smithi196 2024-04-25T01:39:54.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:54.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:54.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:54.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:54.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: Adjusting osd_memory_target on smithi196 to 8544M 2024-04-25T01:39:54.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:54.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:54.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:54.218 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:54.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:54.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:54.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:54.219 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:54.290 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: purged_snaps scrub starts 2024-04-25T01:39:54.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: purged_snaps scrub ok 2024-04-25T01:39:54.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:54.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: osd.5 [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253] boot 2024-04-25T01:39:54.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T01:39:54.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:54.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: Detected new or changed devices on smithi196 2024-04-25T01:39:54.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:54.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:54.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:54.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:54.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: Adjusting osd_memory_target on smithi196 to 8544M 2024-04-25T01:39:54.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:54.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:54.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:54.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:54.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:54.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:54.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:54.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:54.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: purged_snaps scrub starts 2024-04-25T01:39:54.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: purged_snaps scrub ok 2024-04-25T01:39:54.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:54.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: osd.5 [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253] boot 2024-04-25T01:39:54.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T01:39:54.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:39:54.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: Detected new or changed devices on smithi196 2024-04-25T01:39:54.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:54.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:54.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:54.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:39:54.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: Adjusting osd_memory_target on smithi196 to 8544M 2024-04-25T01:39:54.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:54.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:54.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:54.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:54.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:54.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:54.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:54.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:39:55.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.537+0000 7fafbb162700 1 -- 172.21.15.78:0/2435393003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb4103ec0 msgr2=0x7fafb410a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:55.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.537+0000 7fafbb162700 1 --2- 172.21.15.78:0/2435393003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb4103ec0 0x7fafb410a770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fafac00b0a0 tx=0x7fafac00b3b0 comp rx=0 tx=0).stop 2024-04-25T01:39:55.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.537+0000 7fafbb162700 1 -- 172.21.15.78:0/2435393003 shutdown_connections 2024-04-25T01:39:55.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.537+0000 7fafbb162700 1 --2- 172.21.15.78:0/2435393003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb4100540 0x7fafb4100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:55.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.537+0000 7fafbb162700 1 --2- 172.21.15.78:0/2435393003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb4103ec0 0x7fafb410a770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:55.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.537+0000 7fafbb162700 1 --2- 172.21.15.78:0/2435393003 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafb4103520 0x7fafb4103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:55.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.537+0000 7fafbb162700 1 -- 172.21.15.78:0/2435393003 >> 172.21.15.78:0/2435393003 conn(0x7fafb40fbec0 msgr2=0x7fafb40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:55.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.537+0000 7fafbb162700 1 -- 172.21.15.78:0/2435393003 shutdown_connections 2024-04-25T01:39:55.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.537+0000 7fafbb162700 1 -- 172.21.15.78:0/2435393003 wait complete. 2024-04-25T01:39:55.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafbb162700 1 Processor -- start 2024-04-25T01:39:55.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafbb162700 1 -- start start 2024-04-25T01:39:55.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafbb162700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb4100540 0x7fafb4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:55.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafbb162700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafb4103520 0x7fafb4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:55.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafbb162700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb4103ec0 0x7fafb419fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:55.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafbb162700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fafb410df50 con 0x7fafb4100540 2024-04-25T01:39:55.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafbb162700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fafb410ddd0 con 0x7fafb4103520 2024-04-25T01:39:55.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafbb162700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fafb410e0d0 con 0x7fafb4103ec0 2024-04-25T01:39:55.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafb8efe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb4100540 0x7fafb4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:55.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafb8efe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb4100540 0x7fafb4198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48806/0 (socket says 172.21.15.78:48806) 2024-04-25T01:39:55.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafb8efe700 1 -- 172.21.15.78:0/889288591 learned_addr learned my addr 172.21.15.78:0/889288591 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:39:55.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafb96ff700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb4103ec0 0x7fafb419fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:55.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafb3fff700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafb4103520 0x7fafb4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:55.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafb8efe700 1 -- 172.21.15.78:0/889288591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb4103ec0 msgr2=0x7fafb419fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:55.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafb8efe700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb4103ec0 0x7fafb419fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:55.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafb8efe700 1 -- 172.21.15.78:0/889288591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafb4103520 msgr2=0x7fafb4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:55.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafb8efe700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafb4103520 0x7fafb4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:55.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafb8efe700 1 -- 172.21.15.78:0/889288591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb41a0170 con 0x7fafb4100540 2024-04-25T01:39:55.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.538+0000 7fafb3fff700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafb4103520 0x7fafb4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:39:55.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.539+0000 7fafb8efe700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb4100540 0x7fafb4198510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fafa800b530 tx=0x7fafa800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:55.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.539+0000 7fafb1ffb700 1 -- 172.21.15.78:0/889288591 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8015020 con 0x7fafb4100540 2024-04-25T01:39:55.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.539+0000 7fafbb162700 1 -- 172.21.15.78:0/889288591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb41a0460 con 0x7fafb4100540 2024-04-25T01:39:55.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.539+0000 7fafb1ffb700 1 -- 172.21.15.78:0/889288591 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa8011d10 con 0x7fafb4100540 2024-04-25T01:39:55.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.539+0000 7fafb1ffb700 1 -- 172.21.15.78:0/889288591 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa800f2e0 con 0x7fafb4100540 2024-04-25T01:39:55.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.539+0000 7fafbb162700 1 -- 172.21.15.78:0/889288591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb41a0a20 con 0x7fafb4100540 2024-04-25T01:39:55.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.540+0000 7fafbb162700 1 -- 172.21.15.78:0/889288591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb4068f20 con 0x7fafb4100540 2024-04-25T01:39:55.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.541+0000 7fafb1ffb700 1 -- 172.21.15.78:0/889288591 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fafa8018020 con 0x7fafb4100540 2024-04-25T01:39:55.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.541+0000 7fafb1ffb700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fafa40a2c20 0x7fafa40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:55.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.541+0000 7fafb3fff700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fafa40a2c20 0x7fafa40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:55.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.541+0000 7fafb1ffb700 1 -- 172.21.15.78:0/889288591 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7fafa80cd940 con 0x7fafb4100540 2024-04-25T01:39:55.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.542+0000 7fafb3fff700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fafa40a2c20 0x7fafa40a50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fafa00099d0 tx=0x7fafa0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:55.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.546+0000 7fafb1ffb700 1 -- 172.21.15.78:0/889288591 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fafa801f020 con 0x7fafb4100540 2024-04-25T01:39:55.530 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:55 smithi078 ceph-mon[29469]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T01:39:55.530 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:55 smithi078 ceph-mon[29469]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:39:55.608 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:55 smithi196 ceph-mon[32978]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T01:39:55.608 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:55 smithi196 ceph-mon[32978]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:39:55.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:39:55.751+0000 7fafbb162700 1 -- 172.21.15.78:0/889288591 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fafb402df80 con 0x7fafa40a2c20 2024-04-25T01:39:55.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:55 smithi152 ceph-mon[32465]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T01:39:55.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:55 smithi152 ceph-mon[32465]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:39:56.608 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:56 smithi196 ceph-mon[32978]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T01:39:56.608 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:56 smithi196 ceph-mon[32978]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:56.608 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:56 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:39:56.608 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:56 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:39:56.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:56 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:56.662 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:56 smithi078 ceph-mon[29469]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T01:39:56.662 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:56 smithi078 ceph-mon[29469]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:56.662 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:56 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:39:56.662 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:56 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:39:56.663 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:56 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:56.692 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:56 smithi152 ceph-mon[32465]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T01:39:56.692 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:56 smithi152 ceph-mon[32465]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:56.692 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:56 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:39:56.692 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:56 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:39:56.692 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:56 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:57.678 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:57 smithi078 ceph-mon[29469]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T01:39:57.678 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:57 smithi078 ceph-mon[29469]: pgmap v142: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T01:39:57.780 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:57 smithi196 ceph-mon[32978]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T01:39:57.780 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:57 smithi196 ceph-mon[32978]: pgmap v142: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T01:39:57.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:57 smithi152 ceph-mon[32465]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T01:39:57.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:57 smithi152 ceph-mon[32465]: pgmap v142: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T01:39:58.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:58 smithi196 ceph-mon[32978]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T01:39:58.696 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:58 smithi196 ceph-mon[32978]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T01:39:58.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:58 smithi078 ceph-mon[29469]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T01:39:58.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:58 smithi078 ceph-mon[29469]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T01:39:58.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:58 smithi152 ceph-mon[32465]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T01:39:58.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:58 smithi152 ceph-mon[32465]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T01:39:59.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:59 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3793714584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7803a823-c8f7-4015-ba43-f8c0df7eb053"}]: dispatch 2024-04-25T01:39:59.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:59 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3793714584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7803a823-c8f7-4015-ba43-f8c0df7eb053"}]': finished 2024-04-25T01:39:59.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:59 smithi078 ceph-mon[29469]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T01:39:59.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:59 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:39:59.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:59 smithi078 ceph-mon[29469]: pgmap v145: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T01:39:59.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:39:59 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/488971596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:39:59.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:59 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3793714584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7803a823-c8f7-4015-ba43-f8c0df7eb053"}]: dispatch 2024-04-25T01:39:59.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:59 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3793714584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7803a823-c8f7-4015-ba43-f8c0df7eb053"}]': finished 2024-04-25T01:39:59.791 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:59 smithi152 ceph-mon[32465]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T01:39:59.791 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:59 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:39:59.791 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:59 smithi152 ceph-mon[32465]: pgmap v145: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T01:39:59.791 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:39:59 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/488971596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:39:59.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:59 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3793714584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7803a823-c8f7-4015-ba43-f8c0df7eb053"}]: dispatch 2024-04-25T01:39:59.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:59 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3793714584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7803a823-c8f7-4015-ba43-f8c0df7eb053"}]': finished 2024-04-25T01:39:59.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:59 smithi196 ceph-mon[32978]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T01:39:59.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:59 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:39:59.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:59 smithi196 ceph-mon[32978]: pgmap v145: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T01:39:59.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:39:59 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/488971596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:40:00.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:00 smithi152 ceph-mon[32465]: Health detail: HEALTH_WARN Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded 2024-04-25T01:40:00.791 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:00 smithi152 ceph-mon[32465]: [WRN] PG_DEGRADED: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded 2024-04-25T01:40:00.791 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:00 smithi152 ceph-mon[32465]: pg 1.0 is activating+undersized+degraded+remapped, acting [1,2] 2024-04-25T01:40:00.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:00 smithi196 ceph-mon[32978]: Health detail: HEALTH_WARN Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded 2024-04-25T01:40:00.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:00 smithi196 ceph-mon[32978]: [WRN] PG_DEGRADED: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded 2024-04-25T01:40:00.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:00 smithi196 ceph-mon[32978]: pg 1.0 is activating+undersized+degraded+remapped, acting [1,2] 2024-04-25T01:40:00.999 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:00 smithi078 ceph-mon[29469]: Health detail: HEALTH_WARN Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded 2024-04-25T01:40:00.999 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:00 smithi078 ceph-mon[29469]: [WRN] PG_DEGRADED: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded 2024-04-25T01:40:00.999 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:00 smithi078 ceph-mon[29469]: pg 1.0 is activating+undersized+degraded+remapped, acting [1,2] 2024-04-25T01:40:01.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:01 smithi152 ceph-mon[32465]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T01:40:01.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:01 smithi196 ceph-mon[32978]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T01:40:01.999 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:01 smithi078 ceph-mon[29469]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T01:40:02.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:02 smithi152 ceph-mon[32465]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T01:40:02.791 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:02 smithi152 ceph-mon[32465]: Cluster is now healthy 2024-04-25T01:40:02.802 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:02 smithi078 ceph-mon[29469]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T01:40:02.802 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:02 smithi078 ceph-mon[29469]: Cluster is now healthy 2024-04-25T01:40:02.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:02 smithi196 ceph-mon[32978]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T01:40:02.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:02 smithi196 ceph-mon[32978]: Cluster is now healthy 2024-04-25T01:40:03.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:03 smithi078 ceph-mon[29469]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-25T01:40:03.790 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:03 smithi152 ceph-mon[32465]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-25T01:40:03.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:03 smithi196 ceph-mon[32978]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-25T01:40:06.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:05 smithi078 ceph-mon[29469]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T01:40:06.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T01:40:06.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:06.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:05 smithi152 ceph-mon[32465]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T01:40:06.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:05 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T01:40:06.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:05 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:06.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:05 smithi196 ceph-mon[32978]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T01:40:06.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T01:40:06.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:07.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:06 smithi078 ceph-mon[29469]: Deploying daemon osd.6 on smithi078 2024-04-25T01:40:07.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:06 smithi152 ceph-mon[32465]: Deploying daemon osd.6 on smithi078 2024-04-25T01:40:07.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:06 smithi196 ceph-mon[32978]: Deploying daemon osd.6 on smithi078 2024-04-25T01:40:07.939 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:07 smithi078 ceph-mon[29469]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T01:40:08.202 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:07 smithi196 ceph-mon[32978]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T01:40:08.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:07 smithi152 ceph-mon[32465]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T01:40:09.999 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:09 smithi078 ceph-mon[29469]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T01:40:10.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:09 smithi152 ceph-mon[32465]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T01:40:10.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:09 smithi196 ceph-mon[32978]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T01:40:11.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:11 smithi078 ceph-mon[29469]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T01:40:11.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:11 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:11.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:11 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:11.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:11 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:12.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:11 smithi152 ceph-mon[32465]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T01:40:12.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:11 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:12.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:11 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:12.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:11 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:12.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:11 smithi196 ceph-mon[32978]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T01:40:12.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:11 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:12.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:11 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:12.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:11 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:14.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:13 smithi078 ceph-mon[29469]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:40:14.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:13 smithi152 ceph-mon[32465]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:40:14.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:13 smithi196 ceph-mon[32978]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:40:15.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:40:15.156+0000 7fafb1ffb700 1 -- 172.21.15.78:0/889288591 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fafb402df80 con 0x7fafa40a2c20 2024-04-25T01:40:15.141 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 6 on host 'smithi078' 2024-04-25T01:40:15.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:40:15.158+0000 7fafbb162700 1 -- 172.21.15.78:0/889288591 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fafa40a2c20 msgr2=0x7fafa40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:15.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:40:15.158+0000 7fafbb162700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fafa40a2c20 0x7fafa40a50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fafa00099d0 tx=0x7fafa0008040 comp rx=0 tx=0).stop 2024-04-25T01:40:15.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:40:15.158+0000 7fafbb162700 1 -- 172.21.15.78:0/889288591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb4100540 msgr2=0x7fafb4198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:15.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:40:15.158+0000 7fafbb162700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb4100540 0x7fafb4198510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fafa800b530 tx=0x7fafa800b840 comp rx=0 tx=0).stop 2024-04-25T01:40:15.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:40:15.158+0000 7fafbb162700 1 -- 172.21.15.78:0/889288591 shutdown_connections 2024-04-25T01:40:15.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:40:15.158+0000 7fafbb162700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fafa40a2c20 0x7fafa40a50e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:15.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:40:15.158+0000 7fafbb162700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafb4103ec0 0x7fafb419fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:15.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:40:15.158+0000 7fafbb162700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fafb4100540 0x7fafb4198510 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:15.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:40:15.158+0000 7fafbb162700 1 --2- 172.21.15.78:0/889288591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafb4103520 0x7fafb4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:15.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:40:15.158+0000 7fafbb162700 1 -- 172.21.15.78:0/889288591 >> 172.21.15.78:0/889288591 conn(0x7fafb40fbec0 msgr2=0x7fafb41067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:15.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:40:15.159+0000 7fafbb162700 1 -- 172.21.15.78:0/889288591 shutdown_connections 2024-04-25T01:40:15.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:40:15.159+0000 7fafbb162700 1 -- 172.21.15.78:0/889288591 wait complete. 2024-04-25T01:40:15.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:15.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:15.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:15 smithi078 ceph-mon[29469]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:40:15.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:15.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:15.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:15.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:15 smithi152 ceph-mon[32465]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:40:15.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:15.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:15.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:15.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:15 smithi196 ceph-mon[32978]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:40:15.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.610 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:15.610 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:15.866 DEBUG:teuthology.orchestra.run.smithi078:osd.6> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T01:40:15.869 INFO:tasks.cephadm:Deploying osd.7 on smithi152 with /dev/vg_nvme/lv_2... 2024-04-25T01:40:15.869 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T01:40:16.115 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:40:16.404 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:16 smithi078 ceph-mon[29469]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T01:40:16.483 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:16 smithi152 ceph-mon[32465]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T01:40:16.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:16 smithi196 ceph-mon[32978]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T01:40:17.396 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:17 smithi078 ceph-mon[29469]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T01:40:17.396 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:17 smithi078 ceph-mon[29469]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T01:40:17.396 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:17 smithi078 ceph-mon[29469]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T01:40:17.396 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:17.396 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:17 smithi078 ceph-mon[29469]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:40:17.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:17 smithi152 ceph-mon[32465]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T01:40:17.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:17 smithi152 ceph-mon[32465]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T01:40:17.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:17 smithi152 ceph-mon[32465]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T01:40:17.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:17.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:17 smithi152 ceph-mon[32465]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:40:17.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:17 smithi196 ceph-mon[32978]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T01:40:17.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:17 smithi196 ceph-mon[32978]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T01:40:17.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:17 smithi196 ceph-mon[32978]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T01:40:17.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:17.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:17 smithi196 ceph-mon[32978]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:40:17.652 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:40:17 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:40:17.659+0000 7ffbdacf4700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T01:40:17.999 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:40:17 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:40:17.667+0000 7ffbd2adf700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:40:18.246 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T01:40:18.246 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T01:40:18.246 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-04-25T01:40:18.246 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-04-25T01:40:18.246 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134526 s, 779 MB/s 2024-04-25T01:40:18.246 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-04-25T01:40:18.473 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:18 smithi078 ceph-mon[29469]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T01:40:18.474 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:18 smithi078 ceph-mon[29469]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T01:40:18.474 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:18.474 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:18.474 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:18 smithi078 ceph-mon[29469]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' 2024-04-25T01:40:18.474 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:18.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:18 smithi152 ceph-mon[32465]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T01:40:18.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:18 smithi152 ceph-mon[32465]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T01:40:18.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:18.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:18.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:18 smithi152 ceph-mon[32465]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' 2024-04-25T01:40:18.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:18.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:18 smithi196 ceph-mon[32978]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T01:40:18.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:18 smithi196 ceph-mon[32978]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T01:40:18.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:18.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:18.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:18 smithi196 ceph-mon[32978]: from='osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125]' entity='osd.6' 2024-04-25T01:40:18.610 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:18.925 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi152:vg_nvme/lv_2 2024-04-25T01:40:19.171 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:40:19.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: purged_snaps scrub starts 2024-04-25T01:40:19.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: purged_snaps scrub ok 2024-04-25T01:40:19.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125] boot 2024-04-25T01:40:19.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T01:40:19.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:19.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:40:19.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: Detected new or changed devices on smithi078 2024-04-25T01:40:19.502 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:19.502 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:19.502 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:19.502 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:19.502 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:19.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: Adjusting osd_memory_target on smithi078 to 5696M 2024-04-25T01:40:19.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:19.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:19.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:19.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:19.516 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: purged_snaps scrub starts 2024-04-25T01:40:19.516 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: purged_snaps scrub ok 2024-04-25T01:40:19.517 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125] boot 2024-04-25T01:40:19.517 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T01:40:19.517 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:19.517 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:40:19.517 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: Detected new or changed devices on smithi078 2024-04-25T01:40:19.517 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:19.517 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:19.517 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:19.518 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:19.518 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:19.518 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: Adjusting osd_memory_target on smithi078 to 5696M 2024-04-25T01:40:19.518 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:19.518 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:19.518 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:19.519 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:19.610 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: purged_snaps scrub starts 2024-04-25T01:40:19.610 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: purged_snaps scrub ok 2024-04-25T01:40:19.610 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: osd.6 [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125] boot 2024-04-25T01:40:19.610 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T01:40:19.610 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T01:40:19.610 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-25T01:40:19.610 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: Detected new or changed devices on smithi078 2024-04-25T01:40:19.610 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:19.610 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:19.610 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:19.611 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:19.611 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:19.611 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: Adjusting osd_memory_target on smithi078 to 5696M 2024-04-25T01:40:19.611 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:19.611 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:19.611 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:19.611 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:20.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.851+0000 7f5404459700 1 -- 172.21.15.152:0/2415029945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53fc102d30 msgr2=0x7f53fc1095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:20.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.851+0000 7f5404459700 1 --2- 172.21.15.152:0/2415029945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53fc102d30 0x7f53fc1095e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f53f400b0d0 tx=0x7f53f400b3e0 comp rx=0 tx=0).stop 2024-04-25T01:40:20.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.851+0000 7f5404459700 1 -- 172.21.15.152:0/2415029945 shutdown_connections 2024-04-25T01:40:20.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.851+0000 7f5404459700 1 --2- 172.21.15.152:0/2415029945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53fc102d30 0x7f53fc1095e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:20.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.851+0000 7f5404459700 1 --2- 172.21.15.152:0/2415029945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53fc101190 0x7f53fc1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:20.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.851+0000 7f5404459700 1 --2- 172.21.15.152:0/2415029945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53fc102390 0x7f53fc1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:20.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.851+0000 7f5404459700 1 -- 172.21.15.152:0/2415029945 >> 172.21.15.152:0/2415029945 conn(0x7f53fc0fc730 msgr2=0x7f53fc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:20.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.851+0000 7f5404459700 1 -- 172.21.15.152:0/2415029945 shutdown_connections 2024-04-25T01:40:20.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.851+0000 7f5404459700 1 -- 172.21.15.152:0/2415029945 wait complete. 2024-04-25T01:40:20.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.852+0000 7f5404459700 1 Processor -- start 2024-04-25T01:40:20.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.852+0000 7f5404459700 1 -- start start 2024-04-25T01:40:20.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.852+0000 7f5404459700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53fc101190 0x7f53fc06aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:20.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.852+0000 7f5404459700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53fc102390 0x7f53fc19d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:20.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.852+0000 7f5404459700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53fc102d30 0x7f53fc19f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:20.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.852+0000 7f5404459700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc10ce70 con 0x7f53fc102390 2024-04-25T01:40:20.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.852+0000 7f5404459700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc10ccf0 con 0x7f53fc101190 2024-04-25T01:40:20.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.852+0000 7f5404459700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc10cff0 con 0x7f53fc102d30 2024-04-25T01:40:20.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.852+0000 7f54021f5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53fc101190 0x7f53fc06aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:20.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.852+0000 7f54021f5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53fc101190 0x7f53fc06aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37334/0 (socket says 172.21.15.152:37334) 2024-04-25T01:40:20.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.852+0000 7f54021f5700 1 -- 172.21.15.152:0/935363152 learned_addr learned my addr 172.21.15.152:0/935363152 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-25T01:40:20.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f54029f6700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53fc102d30 0x7f53fc19f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:20.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f54019f4700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53fc102390 0x7f53fc19d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:20.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f54021f5700 1 -- 172.21.15.152:0/935363152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53fc102d30 msgr2=0x7f53fc19f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:20.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f54021f5700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53fc102d30 0x7f53fc19f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:20.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f54021f5700 1 -- 172.21.15.152:0/935363152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53fc102390 msgr2=0x7f53fc19d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:20.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f54021f5700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53fc102390 0x7f53fc19d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:20.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f54021f5700 1 -- 172.21.15.152:0/935363152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53fc1a0260 con 0x7f53fc101190 2024-04-25T01:40:20.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f54029f6700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53fc102d30 0x7f53fc19f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:40:20.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f54019f4700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53fc102390 0x7f53fc19d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:40:20.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f54021f5700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53fc101190 0x7f53fc06aa70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53f800b500 tx=0x7f53f800b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:20.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f53f37fe700 1 -- 172.21.15.152:0/935363152 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8015020 con 0x7f53fc101190 2024-04-25T01:40:20.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f53f37fe700 1 -- 172.21.15.152:0/935363152 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f8011d10 con 0x7f53fc101190 2024-04-25T01:40:20.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f5404459700 1 -- 172.21.15.152:0/935363152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53fc197180 con 0x7f53fc101190 2024-04-25T01:40:20.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f53f37fe700 1 -- 172.21.15.152:0/935363152 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f800f2e0 con 0x7f53fc101190 2024-04-25T01:40:20.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.853+0000 7f5404459700 1 -- 172.21.15.152:0/935363152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53fc197790 con 0x7f53fc101190 2024-04-25T01:40:20.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.854+0000 7f5404459700 1 -- 172.21.15.152:0/935363152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53fc068f20 con 0x7f53fc101190 2024-04-25T01:40:20.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.855+0000 7f53f37fe700 1 -- 172.21.15.152:0/935363152 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f53f8018020 con 0x7f53fc101190 2024-04-25T01:40:20.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.856+0000 7f53f37fe700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f53e80a2ca0 0x7f53e80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:20.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.856+0000 7f53f37fe700 1 -- 172.21.15.152:0/935363152 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f53f80cdf30 con 0x7f53fc101190 2024-04-25T01:40:20.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.856+0000 7f54019f4700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f53e80a2ca0 0x7f53e80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:20.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.857+0000 7f54019f4700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f53e80a2ca0 0x7f53e80a5160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f53fc06aff0 tx=0x7f53ec008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:20.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:20.859+0000 7f53f37fe700 1 -- 172.21.15.152:0/935363152 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f53f801f020 con 0x7f53fc101190 2024-04-25T01:40:21.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:21.061+0000 7f5404459700 1 -- 172.21.15.152:0/935363152 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f53fc062670 con 0x7f53e80a2ca0 2024-04-25T01:40:21.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:21 smithi078 ceph-mon[29469]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T01:40:21.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:21 smithi078 ceph-mon[29469]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:21.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:21 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:40:21.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:21 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:40:21.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:21 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:21.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:21 smithi152 ceph-mon[32465]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T01:40:21.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:21 smithi152 ceph-mon[32465]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:21.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:21 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:40:21.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:21 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:40:21.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:21 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:21.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:21 smithi196 ceph-mon[32978]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T01:40:21.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:21 smithi196 ceph-mon[32978]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:21.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:21 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:40:21.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:21 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:40:21.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:21 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:22.291 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:22 smithi152 ceph-mon[32465]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:22.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:22 smithi078 ceph-mon[29469]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:22.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:22 smithi196 ceph-mon[32978]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:23.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:23 smithi078 ceph-mon[29469]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:23.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:23 smithi152 ceph-mon[32465]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:23.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:23 smithi196 ceph-mon[32978]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:24.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:24 smithi078 ceph-mon[29469]: from='client.? 172.21.15.152:0/3427169871' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db92c0f-3343-4fdf-b894-f55c250c741d"}]: dispatch 2024-04-25T01:40:24.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:24 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db92c0f-3343-4fdf-b894-f55c250c741d"}]: dispatch 2024-04-25T01:40:24.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:24 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9db92c0f-3343-4fdf-b894-f55c250c741d"}]': finished 2024-04-25T01:40:24.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:24 smithi078 ceph-mon[29469]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T01:40:24.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:24 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:24.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:24 smithi152 ceph-mon[32465]: from='client.? 172.21.15.152:0/3427169871' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db92c0f-3343-4fdf-b894-f55c250c741d"}]: dispatch 2024-04-25T01:40:24.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:24 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db92c0f-3343-4fdf-b894-f55c250c741d"}]: dispatch 2024-04-25T01:40:24.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:24 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9db92c0f-3343-4fdf-b894-f55c250c741d"}]': finished 2024-04-25T01:40:24.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:24 smithi152 ceph-mon[32465]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T01:40:24.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:24 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:24.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:24 smithi196 ceph-mon[32978]: from='client.? 172.21.15.152:0/3427169871' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db92c0f-3343-4fdf-b894-f55c250c741d"}]: dispatch 2024-04-25T01:40:24.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:24 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db92c0f-3343-4fdf-b894-f55c250c741d"}]: dispatch 2024-04-25T01:40:24.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:24 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9db92c0f-3343-4fdf-b894-f55c250c741d"}]': finished 2024-04-25T01:40:24.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:24 smithi196 ceph-mon[32978]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T01:40:24.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:24 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:25.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:25 smithi078 ceph-mon[29469]: from='client.? 172.21.15.152:0/1569356165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:40:25.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:25 smithi078 ceph-mon[29469]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:25.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:25 smithi152 ceph-mon[32465]: from='client.? 172.21.15.152:0/1569356165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:40:25.541 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:25 smithi152 ceph-mon[32465]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:25.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:25 smithi196 ceph-mon[32978]: from='client.? 172.21.15.152:0/1569356165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:40:25.609 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:25 smithi196 ceph-mon[32978]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:28.041 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:27 smithi152 ceph-mon[32465]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:28.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:27 smithi078 ceph-mon[29469]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:28.312 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:27 smithi196 ceph-mon[32978]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:30.014 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:29 smithi152 ceph-mon[32465]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:30.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:29 smithi078 ceph-mon[29469]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:30.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:29 smithi196 ceph-mon[32978]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:32.080 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:31 smithi152 ceph-mon[32465]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:32.080 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:31 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T01:40:32.080 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:31 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:32.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:31 smithi078 ceph-mon[29469]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:32.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:31 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T01:40:32.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:31 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:32.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:31 smithi196 ceph-mon[32978]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:32.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:31 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T01:40:32.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:31 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:33.041 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:32 smithi152 ceph-mon[32465]: Deploying daemon osd.7 on smithi152 2024-04-25T01:40:33.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:32 smithi078 ceph-mon[29469]: Deploying daemon osd.7 on smithi152 2024-04-25T01:40:33.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:32 smithi196 ceph-mon[32978]: Deploying daemon osd.7 on smithi152 2024-04-25T01:40:34.185 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:33 smithi152 ceph-mon[32465]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:34.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:33 smithi078 ceph-mon[29469]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:34.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:33 smithi196 ceph-mon[32978]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:36.211 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:35 smithi152 ceph-mon[32465]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:36.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:35 smithi078 ceph-mon[29469]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:36.359 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:35 smithi196 ceph-mon[32978]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:38.042 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:38.042 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:38.042 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:38.042 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:37 smithi152 ceph-mon[32465]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:38.110 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:38.110 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:38.110 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:38.110 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:37 smithi196 ceph-mon[32978]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:38.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:38.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:38.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:38.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:37 smithi078 ceph-mon[29469]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:40.170 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:39 smithi152 ceph-mon[32465]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:40.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:39 smithi078 ceph-mon[29469]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:40.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:39 smithi196 ceph-mon[32978]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:40.562 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 7 on host 'smithi152' 2024-04-25T01:40:40.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:40.576+0000 7f53f37fe700 1 -- 172.21.15.152:0/935363152 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f53fc062670 con 0x7f53e80a2ca0 2024-04-25T01:40:40.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:40.580+0000 7f5404459700 1 -- 172.21.15.152:0/935363152 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f53e80a2ca0 msgr2=0x7f53e80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:40.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:40.580+0000 7f5404459700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f53e80a2ca0 0x7f53e80a5160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f53fc06aff0 tx=0x7f53ec008040 comp rx=0 tx=0).stop 2024-04-25T01:40:40.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:40.580+0000 7f5404459700 1 -- 172.21.15.152:0/935363152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53fc101190 msgr2=0x7f53fc06aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:40.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:40.580+0000 7f5404459700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53fc101190 0x7f53fc06aa70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53f800b500 tx=0x7f53f800b8c0 comp rx=0 tx=0).stop 2024-04-25T01:40:40.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:40.580+0000 7f5404459700 1 -- 172.21.15.152:0/935363152 shutdown_connections 2024-04-25T01:40:40.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:40.580+0000 7f5404459700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f53e80a2ca0 0x7f53e80a5160 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:40.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:40.580+0000 7f5404459700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53fc102d30 0x7f53fc19f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:40.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:40.580+0000 7f5404459700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53fc101190 0x7f53fc06aa70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:40.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:40.580+0000 7f5404459700 1 --2- 172.21.15.152:0/935363152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53fc102390 0x7f53fc19d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:40.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:40.580+0000 7f5404459700 1 -- 172.21.15.152:0/935363152 >> 172.21.15.152:0/935363152 conn(0x7f53fc0fc730 msgr2=0x7f53fc105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:40.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:40.580+0000 7f5404459700 1 -- 172.21.15.152:0/935363152 shutdown_connections 2024-04-25T01:40:40.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:40:40.580+0000 7f5404459700 1 -- 172.21.15.152:0/935363152 wait complete. 2024-04-25T01:40:41.165 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:40 smithi152 ceph-mon[32465]: from='osd.7 [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T01:40:41.166 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:40 smithi152 ceph-mon[32465]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T01:40:41.166 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:40 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:41.166 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:40 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:41.166 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:40 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:41.166 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:40 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:41.194 DEBUG:teuthology.orchestra.run.smithi152:osd.7> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T01:40:41.196 INFO:tasks.cephadm:Deploying osd.8 on smithi196 with /dev/vg_nvme/lv_2... 2024-04-25T01:40:41.196 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T01:40:41.223 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:40 smithi196 ceph-mon[32978]: from='osd.7 [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T01:40:41.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:40 smithi196 ceph-mon[32978]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T01:40:41.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:40 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:41.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:40 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:41.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:40 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:41.224 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:40 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:41.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:40 smithi078 ceph-mon[29469]: from='osd.7 [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T01:40:41.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:40 smithi078 ceph-mon[29469]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T01:40:41.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:40 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:41.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:40 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:41.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:40 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:41.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:40 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:41.440 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:40:42.110 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:41 smithi196 ceph-mon[32978]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:42.110 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:41 smithi196 ceph-mon[32978]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T01:40:42.110 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:41 smithi196 ceph-mon[32978]: from='osd.7 [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:40:42.110 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:41 smithi196 ceph-mon[32978]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T01:40:42.110 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:41 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:42.110 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:41 smithi196 ceph-mon[32978]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:40:42.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:41 smithi078 ceph-mon[29469]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:42.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:41 smithi078 ceph-mon[29469]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T01:40:42.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:41 smithi078 ceph-mon[29469]: from='osd.7 [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:40:42.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:41 smithi078 ceph-mon[29469]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T01:40:42.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:42.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:41 smithi078 ceph-mon[29469]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:40:42.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:41 smithi152 ceph-mon[32465]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:42.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:41 smithi152 ceph-mon[32465]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T01:40:42.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:41 smithi152 ceph-mon[32465]: from='osd.7 [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:40:42.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:41 smithi152 ceph-mon[32465]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T01:40:42.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:41 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:42.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:41 smithi152 ceph-mon[32465]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:40:42.293 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:40:41 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:40:41.928+0000 7fd408630700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T01:40:42.293 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:40:41 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:40:41.939+0000 7fd404423700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:40:43.169 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:42 smithi196 ceph-mon[32978]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T01:40:43.169 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:42 smithi196 ceph-mon[32978]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T01:40:43.169 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:42 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:43.169 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:42 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:43.170 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:42 smithi196 ceph-mon[32978]: osd.7 [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251] boot 2024-04-25T01:40:43.170 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:42 smithi196 ceph-mon[32978]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T01:40:43.170 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:42 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:43.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:42 smithi078 ceph-mon[29469]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T01:40:43.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:42 smithi078 ceph-mon[29469]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T01:40:43.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:42 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:43.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:42 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:43.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:42 smithi078 ceph-mon[29469]: osd.7 [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251] boot 2024-04-25T01:40:43.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:42 smithi078 ceph-mon[29469]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T01:40:43.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:42 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:43.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:42 smithi152 ceph-mon[32465]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T01:40:43.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:42 smithi152 ceph-mon[32465]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T01:40:43.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:42 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:43.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:42 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:43.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:42 smithi152 ceph-mon[32465]: osd.7 [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251] boot 2024-04-25T01:40:43.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:42 smithi152 ceph-mon[32465]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T01:40:43.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:42 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T01:40:43.640 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T01:40:43.641 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T01:40:43.641 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-04-25T01:40:43.641 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-04-25T01:40:43.641 INFO:teuthology.orchestra.run.smithi196.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112952 s, 928 MB/s 2024-04-25T01:40:43.641 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-04-25T01:40:44.183 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:43 smithi152 ceph-mon[32465]: purged_snaps scrub starts 2024-04-25T01:40:44.183 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:43 smithi152 ceph-mon[32465]: purged_snaps scrub ok 2024-04-25T01:40:44.183 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:43 smithi152 ceph-mon[32465]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:44.183 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:43 smithi152 ceph-mon[32465]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T01:40:44.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:43 smithi078 ceph-mon[29469]: purged_snaps scrub starts 2024-04-25T01:40:44.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:43 smithi078 ceph-mon[29469]: purged_snaps scrub ok 2024-04-25T01:40:44.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:43 smithi078 ceph-mon[29469]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:44.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:43 smithi078 ceph-mon[29469]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T01:40:44.339 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:43 smithi196 ceph-mon[32978]: purged_snaps scrub starts 2024-04-25T01:40:44.340 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:43 smithi196 ceph-mon[32978]: purged_snaps scrub ok 2024-04-25T01:40:44.340 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:43 smithi196 ceph-mon[32978]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T01:40:44.340 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:43 smithi196 ceph-mon[32978]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T01:40:44.367 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-04-25T01:40:44.642 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:40:45.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: Detected new or changed devices on smithi152 2024-04-25T01:40:45.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:45.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:45.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:45.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: Adjusting osd_memory_target on smithi152 to 5696M 2024-04-25T01:40:45.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:45.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:45.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T01:40:45.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:45.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:45.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:45.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:45 smithi196 ceph-mon[32978]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:45.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: Detected new or changed devices on smithi152 2024-04-25T01:40:45.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:45.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:45.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:45.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: Adjusting osd_memory_target on smithi152 to 5696M 2024-04-25T01:40:45.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:45.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:45.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.501 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T01:40:45.502 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:45.502 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:45.502 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:45.502 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.502 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:45 smithi078 ceph-mon[29469]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:45.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: Detected new or changed devices on smithi152 2024-04-25T01:40:45.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.542 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:45.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:45.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:40:45.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: Adjusting osd_memory_target on smithi152 to 5696M 2024-04-25T01:40:45.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:45.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:45.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T01:40:45.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:45.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:45.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:45.543 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:40:45.544 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:45 smithi152 ceph-mon[32465]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:46.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.343+0000 7ffbabaf9700 1 -- 172.21.15.196:0/1787734057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4103e30 msgr2=0x7ffba410a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:46.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.343+0000 7ffbabaf9700 1 --2- 172.21.15.196:0/1787734057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4103e30 0x7ffba410a6e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00b0a0 tx=0x7ffb9c00b3b0 comp rx=0 tx=0).stop 2024-04-25T01:40:46.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.343+0000 7ffbabaf9700 1 -- 172.21.15.196:0/1787734057 shutdown_connections 2024-04-25T01:40:46.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.343+0000 7ffbabaf9700 1 --2- 172.21.15.196:0/1787734057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4103e30 0x7ffba410a6e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:46.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.343+0000 7ffbabaf9700 1 --2- 172.21.15.196:0/1787734057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffba4102290 0x7ffba41026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:46.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.343+0000 7ffbabaf9700 1 --2- 172.21.15.196:0/1787734057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffba4103490 0x7ffba41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:46.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.343+0000 7ffbabaf9700 1 -- 172.21.15.196:0/1787734057 >> 172.21.15.196:0/1787734057 conn(0x7ffba40fd830 msgr2=0x7ffba40ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:46.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.344+0000 7ffbabaf9700 1 -- 172.21.15.196:0/1787734057 shutdown_connections 2024-04-25T01:40:46.331 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.344+0000 7ffbabaf9700 1 -- 172.21.15.196:0/1787734057 wait complete. 2024-04-25T01:40:46.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.344+0000 7ffbabaf9700 1 Processor -- start 2024-04-25T01:40:46.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.344+0000 7ffbabaf9700 1 -- start start 2024-04-25T01:40:46.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.344+0000 7ffbabaf9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffba4102290 0x7ffba41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:46.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.344+0000 7ffbabaf9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffba4103490 0x7ffba4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:46.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.344+0000 7ffbabaf9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4103e30 0x7ffba419fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:46.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.344+0000 7ffbabaf9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffba410def0 con 0x7ffba4103490 2024-04-25T01:40:46.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.344+0000 7ffbabaf9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffba410dd70 con 0x7ffba4102290 2024-04-25T01:40:46.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.344+0000 7ffbabaf9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffba410e070 con 0x7ffba4103e30 2024-04-25T01:40:46.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.344+0000 7ffbaa096700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4103e30 0x7ffba419fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:46.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffbaa096700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4103e30 0x7ffba419fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:36626/0 (socket says 172.21.15.196:36626) 2024-04-25T01:40:46.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffbaa096700 1 -- 172.21.15.196:0/4019451437 learned_addr learned my addr 172.21.15.196:0/4019451437 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T01:40:46.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffba9094700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffba4103490 0x7ffba4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:46.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffba9895700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffba4102290 0x7ffba41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:46.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffbaa096700 1 -- 172.21.15.196:0/4019451437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffba4102290 msgr2=0x7ffba41984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:46.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffbaa096700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffba4102290 0x7ffba41984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:46.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffbaa096700 1 -- 172.21.15.196:0/4019451437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffba4103490 msgr2=0x7ffba4198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:46.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffbaa096700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffba4103490 0x7ffba4198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:46.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffbaa096700 1 -- 172.21.15.196:0/4019451437 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba41a0130 con 0x7ffba4103e30 2024-04-25T01:40:46.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffba9094700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffba4103490 0x7ffba4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:40:46.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffbaa096700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4103e30 0x7ffba419fa90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00be60 tx=0x7ffb9c007820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:46.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffb9affd700 1 -- 172.21.15.196:0/4019451437 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c021020 con 0x7ffba4103e30 2024-04-25T01:40:46.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffb9affd700 1 -- 172.21.15.196:0/4019451437 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9c005520 con 0x7ffba4103e30 2024-04-25T01:40:46.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffbabaf9700 1 -- 172.21.15.196:0/4019451437 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba41a03c0 con 0x7ffba4103e30 2024-04-25T01:40:46.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffb9affd700 1 -- 172.21.15.196:0/4019451437 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c01ba00 con 0x7ffba4103e30 2024-04-25T01:40:46.333 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.345+0000 7ffbabaf9700 1 -- 172.21.15.196:0/4019451437 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba41a0940 con 0x7ffba4103e30 2024-04-25T01:40:46.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.346+0000 7ffbabaf9700 1 -- 172.21.15.196:0/4019451437 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba4068f20 con 0x7ffba4103e30 2024-04-25T01:40:46.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.347+0000 7ffb9affd700 1 -- 172.21.15.196:0/4019451437 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffb9c024020 con 0x7ffba4103e30 2024-04-25T01:40:46.334 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.348+0000 7ffb9affd700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ffb900a2ca0 0x7ffb900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:46.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.348+0000 7ffb9affd700 1 -- 172.21.15.196:0/4019451437 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7ffb9c00f020 con 0x7ffba4103e30 2024-04-25T01:40:46.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.348+0000 7ffba9895700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ffb900a2ca0 0x7ffb900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:46.335 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.349+0000 7ffba9895700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ffb900a2ca0 0x7ffb900a5160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffba0006fd0 tx=0x7ffba0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:46.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.351+0000 7ffb9affd700 1 -- 172.21.15.196:0/4019451437 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffb9c09d660 con 0x7ffba4103e30 2024-04-25T01:40:46.543 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:40:46.558+0000 7ffbabaf9700 1 -- 172.21.15.196:0/4019451437 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ffba4062670 con 0x7ffb900a2ca0 2024-04-25T01:40:46.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:46 smithi078 ceph-mon[29469]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T01:40:46.792 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:46 smithi152 ceph-mon[32465]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T01:40:46.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:46 smithi196 ceph-mon[32978]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T01:40:47.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:47 smithi078 ceph-mon[29469]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T01:40:47.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:47 smithi078 ceph-mon[29469]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:47.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:40:47.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:40:47.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:47.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:47 smithi078 ceph-mon[29469]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:47.792 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:47 smithi152 ceph-mon[32465]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T01:40:47.792 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:47 smithi152 ceph-mon[32465]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:47.792 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:47 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:40:47.792 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:47 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:40:47.793 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:47 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:47.793 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:47 smithi152 ceph-mon[32465]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:47.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:47 smithi196 ceph-mon[32978]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T01:40:47.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:47 smithi196 ceph-mon[32978]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:47.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:47 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:40:47.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:47 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:40:47.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:47 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:47.861 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:47 smithi196 ceph-mon[32978]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:50.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:49 smithi078 ceph-mon[29469]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:50.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:49 smithi078 ceph-mon[29469]: from='client.? 172.21.15.196:0/2150981169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28f5d9f1-e260-43b1-9741-2717211085e6"}]: dispatch 2024-04-25T01:40:50.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:49 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28f5d9f1-e260-43b1-9741-2717211085e6"}]: dispatch 2024-04-25T01:40:50.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:49 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28f5d9f1-e260-43b1-9741-2717211085e6"}]': finished 2024-04-25T01:40:50.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:49 smithi078 ceph-mon[29469]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T01:40:50.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:40:50.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:49 smithi152 ceph-mon[32465]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:50.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:49 smithi152 ceph-mon[32465]: from='client.? 172.21.15.196:0/2150981169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28f5d9f1-e260-43b1-9741-2717211085e6"}]: dispatch 2024-04-25T01:40:50.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:49 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28f5d9f1-e260-43b1-9741-2717211085e6"}]: dispatch 2024-04-25T01:40:50.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:49 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28f5d9f1-e260-43b1-9741-2717211085e6"}]': finished 2024-04-25T01:40:50.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:49 smithi152 ceph-mon[32465]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T01:40:50.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:40:50.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:49 smithi196 ceph-mon[32978]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:50.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:49 smithi196 ceph-mon[32978]: from='client.? 172.21.15.196:0/2150981169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28f5d9f1-e260-43b1-9741-2717211085e6"}]: dispatch 2024-04-25T01:40:50.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:49 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28f5d9f1-e260-43b1-9741-2717211085e6"}]: dispatch 2024-04-25T01:40:50.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:49 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28f5d9f1-e260-43b1-9741-2717211085e6"}]': finished 2024-04-25T01:40:50.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:49 smithi196 ceph-mon[32978]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T01:40:50.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:40:51.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:50 smithi078 ceph-mon[29469]: from='client.? 172.21.15.196:0/1854147108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:40:51.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:50 smithi152 ceph-mon[32465]: from='client.? 172.21.15.196:0/1854147108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:40:51.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:50 smithi196 ceph-mon[32978]: from='client.? 172.21.15.196:0/1854147108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:40:52.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:51 smithi078 ceph-mon[29469]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:52.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:51 smithi152 ceph-mon[32465]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:52.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:51 smithi196 ceph-mon[32978]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:54.221 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:53 smithi196 ceph-mon[32978]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:54.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:53 smithi078 ceph-mon[29469]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:54.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:53 smithi152 ceph-mon[32465]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:56.245 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:55 smithi196 ceph-mon[32978]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:56.250 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:55 smithi078 ceph-mon[29469]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:56.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:55 smithi152 ceph-mon[32465]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:57.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:56 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T01:40:57.185 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:56 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:57.194 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:56 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T01:40:57.194 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:56 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:57.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:56 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T01:40:57.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:56 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:58.204 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:57 smithi196 ceph-mon[32978]: Deploying daemon osd.8 on smithi196 2024-04-25T01:40:58.204 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:57 smithi196 ceph-mon[32978]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:58.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:57 smithi078 ceph-mon[29469]: Deploying daemon osd.8 on smithi196 2024-04-25T01:40:58.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:57 smithi078 ceph-mon[29469]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:40:58.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:57 smithi152 ceph-mon[32465]: Deploying daemon osd.8 on smithi196 2024-04-25T01:40:58.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:57 smithi152 ceph-mon[32465]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:00.110 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:40:59 smithi196 ceph-mon[32978]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:00.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:40:59 smithi078 ceph-mon[29469]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:00.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:40:59 smithi152 ceph-mon[32465]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:02.105 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:01 smithi196 ceph-mon[32978]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:02.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:01 smithi078 ceph-mon[29469]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:02.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:01 smithi152 ceph-mon[32465]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:03.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:03.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:03.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:03.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:03 smithi078 ceph-mon[29469]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:03.793 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:03.793 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:03.793 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:03.793 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:03 smithi152 ceph-mon[32465]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:03.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:03.861 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:03.861 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:03.861 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:03 smithi196 ceph-mon[32978]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:06.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:05 smithi196 ceph-mon[32978]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:06.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:06.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:06.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:06.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:06.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:05 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:06.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:05 smithi078 ceph-mon[29469]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:06.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:06.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:06.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:06.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:06.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:05 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:06.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:05 smithi152 ceph-mon[32465]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:06.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:05 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:06.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:05 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:06.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:05 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:06.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:05 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:06.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:05 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:06.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:41:06.624+0000 7ffb9affd700 1 -- 172.21.15.196:0/4019451437 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffba4062670 con 0x7ffb900a2ca0 2024-04-25T01:41:06.611 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 8 on host 'smithi196' 2024-04-25T01:41:06.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:41:06.626+0000 7ffbabaf9700 1 -- 172.21.15.196:0/4019451437 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ffb900a2ca0 msgr2=0x7ffb900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:41:06.626+0000 7ffbabaf9700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ffb900a2ca0 0x7ffb900a5160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffba0006fd0 tx=0x7ffba0008040 comp rx=0 tx=0).stop 2024-04-25T01:41:06.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:41:06.626+0000 7ffbabaf9700 1 -- 172.21.15.196:0/4019451437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4103e30 msgr2=0x7ffba419fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:06.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:41:06.626+0000 7ffbabaf9700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4103e30 0x7ffba419fa90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00be60 tx=0x7ffb9c007820 comp rx=0 tx=0).stop 2024-04-25T01:41:06.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:41:06.626+0000 7ffbabaf9700 1 -- 172.21.15.196:0/4019451437 shutdown_connections 2024-04-25T01:41:06.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:41:06.626+0000 7ffbabaf9700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ffb900a2ca0 0x7ffb900a5160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:41:06.626+0000 7ffbabaf9700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba4103e30 0x7ffba419fa90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:41:06.626+0000 7ffbabaf9700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffba4102290 0x7ffba41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:41:06.626+0000 7ffbabaf9700 1 --2- 172.21.15.196:0/4019451437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffba4103490 0x7ffba4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:06.613 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:41:06.626+0000 7ffbabaf9700 1 -- 172.21.15.196:0/4019451437 >> 172.21.15.196:0/4019451437 conn(0x7ffba40fd830 msgr2=0x7ffba4106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:06.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:41:06.626+0000 7ffbabaf9700 1 -- 172.21.15.196:0/4019451437 shutdown_connections 2024-04-25T01:41:06.614 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:41:06.626+0000 7ffbabaf9700 1 -- 172.21.15.196:0/4019451437 wait complete. 2024-04-25T01:41:07.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:06 smithi078 ceph-mon[29469]: from='osd.8 [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T01:41:07.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:06 smithi078 ceph-mon[29469]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T01:41:07.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:06 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:07.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:06 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:07.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:06 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:07.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:06 smithi196 ceph-mon[32978]: from='osd.8 [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T01:41:07.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:06 smithi196 ceph-mon[32978]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T01:41:07.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:06 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:07.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:06 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:07.272 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:06 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:07.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:06 smithi152 ceph-mon[32465]: from='osd.8 [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T01:41:07.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:06 smithi152 ceph-mon[32465]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T01:41:07.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:06 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:07.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:06 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:07.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:06 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:07.299 DEBUG:teuthology.orchestra.run.smithi196:osd.8> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T01:41:07.301 INFO:tasks.cephadm:Deploying osd.9 on smithi078 with /dev/vg_nvme/lv_1... 2024-04-25T01:41:07.301 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T01:41:07.546 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:41:07.929 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:07 smithi078 ceph-mon[29469]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:07.929 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:07 smithi078 ceph-mon[29469]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T01:41:07.929 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:07 smithi078 ceph-mon[29469]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T01:41:07.929 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:07 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:07.929 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:07 smithi078 ceph-mon[29469]: from='osd.8 [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:41:07.930 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:07 smithi078 ceph-mon[29469]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:41:08.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:07 smithi196 ceph-mon[32978]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:08.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:07 smithi196 ceph-mon[32978]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T01:41:08.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:07 smithi196 ceph-mon[32978]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T01:41:08.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:07 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:08.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:07 smithi196 ceph-mon[32978]: from='osd.8 [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:41:08.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:07 smithi196 ceph-mon[32978]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:41:08.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:07 smithi152 ceph-mon[32465]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:08.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:07 smithi152 ceph-mon[32465]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T01:41:08.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:07 smithi152 ceph-mon[32465]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T01:41:08.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:07 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:08.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:07 smithi152 ceph-mon[32465]: from='osd.8 [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:41:08.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:07 smithi152 ceph-mon[32465]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:41:08.759 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:41:08 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:41:08.373+0000 7f855375b700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T01:41:08.759 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:41:08 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:41:08.384+0000 7f854b546700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:41:09.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:08 smithi078 ceph-mon[29469]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-25T01:41:09.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:08 smithi078 ceph-mon[29469]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T01:41:09.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:08 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:09.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:08 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:09.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:08 smithi078 ceph-mon[29469]: from='osd.8 ' entity='osd.8' 2024-04-25T01:41:09.066 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:08 smithi196 ceph-mon[32978]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-25T01:41:09.066 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:08 smithi196 ceph-mon[32978]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T01:41:09.066 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:08 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:09.067 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:08 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:09.067 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:08 smithi196 ceph-mon[32978]: from='osd.8 ' entity='osd.8' 2024-04-25T01:41:09.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:08 smithi152 ceph-mon[32465]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-25T01:41:09.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:08 smithi152 ceph-mon[32465]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T01:41:09.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:08 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:09.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:08 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:09.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:08 smithi152 ceph-mon[32465]: from='osd.8 ' entity='osd.8' 2024-04-25T01:41:09.601 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T01:41:09.601 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T01:41:09.601 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-04-25T01:41:09.601 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-04-25T01:41:09.601 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115578 s, 907 MB/s 2024-04-25T01:41:09.601 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-04-25T01:41:10.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:09 smithi196 ceph-mon[32978]: purged_snaps scrub starts 2024-04-25T01:41:10.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:09 smithi196 ceph-mon[32978]: purged_snaps scrub ok 2024-04-25T01:41:10.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:09 smithi196 ceph-mon[32978]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:10.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:10.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:09 smithi196 ceph-mon[32978]: osd.8 [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426] boot 2024-04-25T01:41:10.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:09 smithi196 ceph-mon[32978]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T01:41:10.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:10.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:09 smithi078 ceph-mon[29469]: purged_snaps scrub starts 2024-04-25T01:41:10.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:09 smithi078 ceph-mon[29469]: purged_snaps scrub ok 2024-04-25T01:41:10.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:09 smithi078 ceph-mon[29469]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:10.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:10.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:09 smithi078 ceph-mon[29469]: osd.8 [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426] boot 2024-04-25T01:41:10.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:09 smithi078 ceph-mon[29469]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T01:41:10.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:10.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:09 smithi152 ceph-mon[32465]: purged_snaps scrub starts 2024-04-25T01:41:10.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:09 smithi152 ceph-mon[32465]: purged_snaps scrub ok 2024-04-25T01:41:10.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:09 smithi152 ceph-mon[32465]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T01:41:10.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:10.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:09 smithi152 ceph-mon[32465]: osd.8 [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426] boot 2024-04-25T01:41:10.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:09 smithi152 ceph-mon[32465]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T01:41:10.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T01:41:10.385 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_1 2024-04-25T01:41:10.621 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:41:10.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:10 smithi078 ceph-mon[29469]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T01:41:10.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:10 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:10.994 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:10 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:10.995 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:10 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:10.995 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:10 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:10.995 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:10 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:10.995 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:10 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:10.995 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:10 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:10.995 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:10 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:10.995 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:10 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:11.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:10 smithi152 ceph-mon[32465]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T01:41:11.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:10 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:11.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:10 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:11.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:10 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:11.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:10 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:11.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:10 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:11.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:10 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:11.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:10 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:11.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:10 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:11.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:10 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:11.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:10 smithi196 ceph-mon[32978]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T01:41:11.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:10 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:11.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:10 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:11.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:10 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:11.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:10 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:11.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:10 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:11.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:10 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:11.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:10 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:11.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:10 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:11.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:10 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:12.093 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:11 smithi078 ceph-mon[29469]: Detected new or changed devices on smithi196 2024-04-25T01:41:12.093 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:11 smithi078 ceph-mon[29469]: Adjusting osd_memory_target on smithi196 to 5696M 2024-04-25T01:41:12.093 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:11 smithi078 ceph-mon[29469]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:12.093 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:11 smithi078 ceph-mon[29469]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T01:41:12.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:11 smithi152 ceph-mon[32465]: Detected new or changed devices on smithi196 2024-04-25T01:41:12.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:11 smithi152 ceph-mon[32465]: Adjusting osd_memory_target on smithi196 to 5696M 2024-04-25T01:41:12.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:11 smithi152 ceph-mon[32465]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:12.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:11 smithi152 ceph-mon[32465]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T01:41:12.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.361+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1611543407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdec107f80 msgr2=0x7fbdec10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:12.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.361+0000 7fbdf1e92700 1 --2- 172.21.15.78:0/1611543407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdec107f80 0x7fbdec10a3f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbde400b0a0 tx=0x7fbde400b3b0 comp rx=0 tx=0).stop 2024-04-25T01:41:12.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.361+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1611543407 shutdown_connections 2024-04-25T01:41:12.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.361+0000 7fbdf1e92700 1 --2- 172.21.15.78:0/1611543407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdec102d20 0x7fbdec105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:12.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.361+0000 7fbdf1e92700 1 --2- 172.21.15.78:0/1611543407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdec107f80 0x7fbdec10a3f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:12.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.361+0000 7fbdf1e92700 1 --2- 172.21.15.78:0/1611543407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdec105650 0x7fbdec107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:12.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.361+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1611543407 >> 172.21.15.78:0/1611543407 conn(0x7fbdec0fc730 msgr2=0x7fbdec0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:12.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.361+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1611543407 shutdown_connections 2024-04-25T01:41:12.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.361+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1611543407 wait complete. 2024-04-25T01:41:12.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.362+0000 7fbdf1e92700 1 Processor -- start 2024-04-25T01:41:12.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.362+0000 7fbdf1e92700 1 -- start start 2024-04-25T01:41:12.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.362+0000 7fbdf1e92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdec102d20 0x7fbdec1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:12.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.362+0000 7fbdf1e92700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdec105650 0x7fbdec1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:12.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.362+0000 7fbdf1e92700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdec107f80 0x7fbdec19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:12.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.362+0000 7fbdf1e92700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec10dec0 con 0x7fbdec102d20 2024-04-25T01:41:12.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.362+0000 7fbdf1e92700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec10dd40 con 0x7fbdec107f80 2024-04-25T01:41:12.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.362+0000 7fbdf1e92700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec10e040 con 0x7fbdec105650 2024-04-25T01:41:12.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.362+0000 7fbdeb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdec102d20 0x7fbdec1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:12.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.362+0000 7fbdeb7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdec102d20 0x7fbdec1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58458/0 (socket says 172.21.15.78:58458) 2024-04-25T01:41:12.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.362+0000 7fbdeb7fe700 1 -- 172.21.15.78:0/1265968839 learned_addr learned my addr 172.21.15.78:0/1265968839 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:41:12.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.362+0000 7fbdebfff700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdec107f80 0x7fbdec19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:12.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.362+0000 7fbdeaffd700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdec105650 0x7fbdec1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:12.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.363+0000 7fbdeb7fe700 1 -- 172.21.15.78:0/1265968839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdec105650 msgr2=0x7fbdec1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:12.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.363+0000 7fbdeb7fe700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdec105650 0x7fbdec1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:12.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.363+0000 7fbdeb7fe700 1 -- 172.21.15.78:0/1265968839 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdec107f80 msgr2=0x7fbdec19fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:12.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.363+0000 7fbdeb7fe700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdec107f80 0x7fbdec19fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:12.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.363+0000 7fbdeb7fe700 1 -- 172.21.15.78:0/1265968839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdec1a0110 con 0x7fbdec102d20 2024-04-25T01:41:12.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.363+0000 7fbdeaffd700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdec105650 0x7fbdec1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:12.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.363+0000 7fbdeb7fe700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdec102d20 0x7fbdec1984b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00b560 tx=0x7fbddc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:12.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.363+0000 7fbde8ff9700 1 -- 172.21.15.78:0/1265968839 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc015020 con 0x7fbdec102d20 2024-04-25T01:41:12.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.363+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1265968839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdec1a0400 con 0x7fbdec102d20 2024-04-25T01:41:12.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.363+0000 7fbde8ff9700 1 -- 172.21.15.78:0/1265968839 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbddc011d10 con 0x7fbdec102d20 2024-04-25T01:41:12.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.363+0000 7fbde8ff9700 1 -- 172.21.15.78:0/1265968839 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc010e50 con 0x7fbdec102d20 2024-04-25T01:41:12.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.363+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1265968839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdec1a0940 con 0x7fbdec102d20 2024-04-25T01:41:12.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.364+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1265968839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdec068f20 con 0x7fbdec102d20 2024-04-25T01:41:12.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.365+0000 7fbde8ff9700 1 -- 172.21.15.78:0/1265968839 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbddc018020 con 0x7fbdec102d20 2024-04-25T01:41:12.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.366+0000 7fbde8ff9700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbdd40a2dc0 0x7fbdd40a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:12.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.366+0000 7fbdeaffd700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbdd40a2dc0 0x7fbdd40a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:12.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.366+0000 7fbde8ff9700 1 -- 172.21.15.78:0/1265968839 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7fbddc0ce410 con 0x7fbdec102d20 2024-04-25T01:41:12.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.366+0000 7fbdeaffd700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbdd40a2dc0 0x7fbdd40a5280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbde0005950 tx=0x7fbde00058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:12.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.369+0000 7fbde8ff9700 1 -- 172.21.15.78:0/1265968839 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbddc01f020 con 0x7fbdec102d20 2024-04-25T01:41:12.360 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:11 smithi196 ceph-mon[32978]: Detected new or changed devices on smithi196 2024-04-25T01:41:12.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:11 smithi196 ceph-mon[32978]: Adjusting osd_memory_target on smithi196 to 5696M 2024-04-25T01:41:12.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:11 smithi196 ceph-mon[32978]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:12.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:11 smithi196 ceph-mon[32978]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T01:41:12.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:12.576+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1265968839 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbdec062670 con 0x7fbdd40a2dc0 2024-04-25T01:41:13.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:12 smithi078 ceph-mon[29469]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T01:41:13.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:12 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:41:13.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:12 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:41:13.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:12 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:13.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:12 smithi152 ceph-mon[32465]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T01:41:13.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:12 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:41:13.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:12 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:41:13.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:12 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:13.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:12 smithi196 ceph-mon[32978]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T01:41:13.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:12 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:41:13.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:12 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:41:13.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:12 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:14.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:13 smithi078 ceph-mon[29469]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:14.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:13 smithi078 ceph-mon[29469]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:14.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:13 smithi078 ceph-mon[29469]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T01:41:14.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:13 smithi152 ceph-mon[32465]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:14.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:13 smithi152 ceph-mon[32465]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:14.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:13 smithi152 ceph-mon[32465]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T01:41:14.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:13 smithi196 ceph-mon[32978]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:14.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:13 smithi196 ceph-mon[32978]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:14.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:13 smithi196 ceph-mon[32978]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T01:41:16.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:15 smithi078 ceph-mon[29469]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T01:41:16.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:15 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3538559391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07876526-a50e-4031-ad91-552cd02a1b6e"}]: dispatch 2024-04-25T01:41:16.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:15 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3538559391' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07876526-a50e-4031-ad91-552cd02a1b6e"}]': finished 2024-04-25T01:41:16.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:15 smithi078 ceph-mon[29469]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T01:41:16.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:16.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:15 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2915730024' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:41:16.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:15 smithi152 ceph-mon[32465]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T01:41:16.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:15 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3538559391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07876526-a50e-4031-ad91-552cd02a1b6e"}]: dispatch 2024-04-25T01:41:16.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:15 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3538559391' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07876526-a50e-4031-ad91-552cd02a1b6e"}]': finished 2024-04-25T01:41:16.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:15 smithi152 ceph-mon[32465]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T01:41:16.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:16.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:15 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2915730024' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:41:16.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:15 smithi196 ceph-mon[32978]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T01:41:16.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:15 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3538559391' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07876526-a50e-4031-ad91-552cd02a1b6e"}]: dispatch 2024-04-25T01:41:16.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:15 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3538559391' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07876526-a50e-4031-ad91-552cd02a1b6e"}]': finished 2024-04-25T01:41:16.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:15 smithi196 ceph-mon[32978]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T01:41:16.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:16.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:15 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2915730024' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:41:18.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:17 smithi078 ceph-mon[29469]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-25T01:41:18.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:17 smithi152 ceph-mon[32465]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-25T01:41:18.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:17 smithi196 ceph-mon[32978]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-25T01:41:20.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:19 smithi078 ceph-mon[29469]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T01:41:20.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:19 smithi152 ceph-mon[32465]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T01:41:20.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:19 smithi196 ceph-mon[32978]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T01:41:22.198 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:21 smithi078 ceph-mon[29469]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T01:41:22.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:21 smithi152 ceph-mon[32465]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T01:41:22.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:21 smithi196 ceph-mon[32978]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T01:41:24.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:23 smithi078 ceph-mon[29469]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:24.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:23 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T01:41:24.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:23 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:24.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:23 smithi078 ceph-mon[29469]: Deploying daemon osd.9 on smithi078 2024-04-25T01:41:24.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:23 smithi152 ceph-mon[32465]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:24.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:23 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T01:41:24.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:23 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:24.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:23 smithi152 ceph-mon[32465]: Deploying daemon osd.9 on smithi078 2024-04-25T01:41:24.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:23 smithi196 ceph-mon[32978]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:24.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:23 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T01:41:24.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:23 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:24.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:23 smithi196 ceph-mon[32978]: Deploying daemon osd.9 on smithi078 2024-04-25T01:41:26.144 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:25 smithi078 ceph-mon[29469]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:26.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:25 smithi152 ceph-mon[32465]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:26.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:25 smithi196 ceph-mon[32978]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:28.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:27 smithi078 ceph-mon[29469]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:28.293 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:27 smithi152 ceph-mon[32465]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:28.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:27 smithi196 ceph-mon[32978]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:30.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:29 smithi078 ceph-mon[29469]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:30.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:30.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:30.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:30.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:29 smithi152 ceph-mon[32465]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:30.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:30.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:30.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:30.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:29 smithi196 ceph-mon[32978]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:30.356 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:30.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:30.357 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:32.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:31 smithi078 ceph-mon[29469]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:32.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:31 smithi152 ceph-mon[32465]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:32.361 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:31 smithi196 ceph-mon[32978]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:32.721 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 9 on host 'smithi078' 2024-04-25T01:41:32.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:32.734+0000 7fbde8ff9700 1 -- 172.21.15.78:0/1265968839 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbdec062670 con 0x7fbdd40a2dc0 2024-04-25T01:41:32.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:32.737+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1265968839 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbdd40a2dc0 msgr2=0x7fbdd40a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:32.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:32.737+0000 7fbdf1e92700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbdd40a2dc0 0x7fbdd40a5280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbde0005950 tx=0x7fbde00058e0 comp rx=0 tx=0).stop 2024-04-25T01:41:32.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:32.737+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1265968839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdec102d20 msgr2=0x7fbdec1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:32.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:32.737+0000 7fbdf1e92700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdec102d20 0x7fbdec1984b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00b560 tx=0x7fbddc00b870 comp rx=0 tx=0).stop 2024-04-25T01:41:32.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:32.737+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1265968839 shutdown_connections 2024-04-25T01:41:32.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:32.737+0000 7fbdf1e92700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbdd40a2dc0 0x7fbdd40a5280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:32.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:32.737+0000 7fbdf1e92700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbdec107f80 0x7fbdec19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:32.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:32.737+0000 7fbdf1e92700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdec102d20 0x7fbdec1984b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:32.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:32.737+0000 7fbdf1e92700 1 --2- 172.21.15.78:0/1265968839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdec105650 0x7fbdec1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:32.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:32.737+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1265968839 >> 172.21.15.78:0/1265968839 conn(0x7fbdec0fc730 msgr2=0x7fbdec1035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:32.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:32.737+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1265968839 shutdown_connections 2024-04-25T01:41:32.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:41:32.737+0000 7fbdf1e92700 1 -- 172.21.15.78:0/1265968839 wait complete. 2024-04-25T01:41:33.431 DEBUG:teuthology.orchestra.run.smithi078:osd.9> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T01:41:33.434 INFO:tasks.cephadm:Deploying osd.10 on smithi152 with /dev/vg_nvme/lv_1... 2024-04-25T01:41:33.434 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T01:41:33.684 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:41:33.968 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:33 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:33.968 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:33 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:33.968 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:33 smithi152 ceph-mon[32465]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T01:41:33.968 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:33 smithi152 ceph-mon[32465]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:33.968 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:33 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:33.968 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:33 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:34.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:33 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:34.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:33 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:34.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:33 smithi078 ceph-mon[29469]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T01:41:34.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:33 smithi078 ceph-mon[29469]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:34.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:33 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:34.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:33 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:34.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:33 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:34.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:33 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:34.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:33 smithi196 ceph-mon[32978]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T01:41:34.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:33 smithi196 ceph-mon[32978]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:34.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:33 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:34.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:33 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:34.992 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:34 smithi152 ceph-mon[32465]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T01:41:34.992 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:34 smithi152 ceph-mon[32465]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T01:41:34.992 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:34 smithi152 ceph-mon[32465]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T01:41:34.992 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:34 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:35.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:34 smithi078 ceph-mon[29469]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T01:41:35.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:34 smithi078 ceph-mon[29469]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T01:41:35.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:34 smithi078 ceph-mon[29469]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T01:41:35.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:34 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:35.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:34 smithi196 ceph-mon[32978]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T01:41:35.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:34 smithi196 ceph-mon[32978]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T01:41:35.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:34 smithi196 ceph-mon[32978]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-04-25T01:41:35.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:34 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:35.474 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:41:35 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:41:35.160+0000 7f20dba47700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T01:41:35.474 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:41:35 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:41:35.173+0000 7f20d3832700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:41:35.680 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T01:41:35.680 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T01:41:35.680 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-04-25T01:41:35.680 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-04-25T01:41:35.680 INFO:teuthology.orchestra.run.smithi152.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113484 s, 924 MB/s 2024-04-25T01:41:35.680 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-04-25T01:41:36.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:35 smithi078 ceph-mon[29469]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T01:41:36.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:35 smithi078 ceph-mon[29469]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T01:41:36.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:35 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:36.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:35 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:36.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:35 smithi078 ceph-mon[29469]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:36.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:35 smithi078 ceph-mon[29469]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' 2024-04-25T01:41:36.015 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:35 smithi152 ceph-mon[32465]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T01:41:36.015 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:35 smithi152 ceph-mon[32465]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T01:41:36.016 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:35 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:36.016 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:35 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:36.016 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:35 smithi152 ceph-mon[32465]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:36.016 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:35 smithi152 ceph-mon[32465]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' 2024-04-25T01:41:36.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:35 smithi196 ceph-mon[32978]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-04-25T01:41:36.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:35 smithi196 ceph-mon[32978]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T01:41:36.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:35 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:36.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:35 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:36.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:35 smithi196 ceph-mon[32978]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-25T01:41:36.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:35 smithi196 ceph-mon[32978]: from='osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277]' entity='osd.9' 2024-04-25T01:41:36.286 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi152:vg_nvme/lv_1 2024-04-25T01:41:36.530 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:41:36.878 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:36 smithi152 ceph-mon[32465]: purged_snaps scrub starts 2024-04-25T01:41:36.879 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:36 smithi152 ceph-mon[32465]: purged_snaps scrub ok 2024-04-25T01:41:36.879 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:36 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:36.879 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:36 smithi152 ceph-mon[32465]: osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277] boot 2024-04-25T01:41:36.879 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:36 smithi152 ceph-mon[32465]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T01:41:36.879 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:36 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:36.955 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:36 smithi078 ceph-mon[29469]: purged_snaps scrub starts 2024-04-25T01:41:36.955 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:36 smithi078 ceph-mon[29469]: purged_snaps scrub ok 2024-04-25T01:41:36.955 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:36 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:36.955 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:36 smithi078 ceph-mon[29469]: osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277] boot 2024-04-25T01:41:36.955 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:36 smithi078 ceph-mon[29469]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T01:41:36.955 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:36 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:37.111 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:36 smithi196 ceph-mon[32978]: purged_snaps scrub starts 2024-04-25T01:41:37.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:36 smithi196 ceph-mon[32978]: purged_snaps scrub ok 2024-04-25T01:41:37.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:36 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:37.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:36 smithi196 ceph-mon[32978]: osd.9 [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277] boot 2024-04-25T01:41:37.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:36 smithi196 ceph-mon[32978]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T01:41:37.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:36 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T01:41:37.904 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:37.905 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: Detected new or changed devices on smithi078 2024-04-25T01:41:37.905 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:37.905 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:37.905 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:37.905 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:37.905 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:37.905 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:37.905 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:37.905 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:37.905 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:37.905 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:37.905 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T01:41:37.905 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:37.906 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:37.906 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:37.906 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:38.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:38.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: Detected new or changed devices on smithi078 2024-04-25T01:41:38.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:38.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:38.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:38.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:38.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:38.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:38.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:38.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:38.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:38.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:38.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T01:41:38.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:38.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:38.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:38.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:38.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:38.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: Detected new or changed devices on smithi078 2024-04-25T01:41:38.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:38.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:38.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:38.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:38.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:38.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:41:38.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:38.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:38.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:38.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:38.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T01:41:38.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:38.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:38.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:38.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:38.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.146+0000 7fd307934700 1 -- 172.21.15.152:0/874659117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd300100630 msgr2=0x7fd300100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:38.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.146+0000 7fd307934700 1 --2- 172.21.15.152:0/874659117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd300100630 0x7fd300100a50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009990 tx=0x7fd2fc009ca0 comp rx=0 tx=0).stop 2024-04-25T01:41:38.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.146+0000 7fd307934700 1 -- 172.21.15.152:0/874659117 shutdown_connections 2024-04-25T01:41:38.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.146+0000 7fd307934700 1 --2- 172.21.15.152:0/874659117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd300103fb0 0x7fd30010a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:38.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.146+0000 7fd307934700 1 --2- 172.21.15.152:0/874659117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd300100630 0x7fd300100a50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:38.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.146+0000 7fd307934700 1 --2- 172.21.15.152:0/874659117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd300103610 0x7fd300103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:38.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.147+0000 7fd307934700 1 -- 172.21.15.152:0/874659117 >> 172.21.15.152:0/874659117 conn(0x7fd3000fbf90 msgr2=0x7fd3000fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:38.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.147+0000 7fd307934700 1 -- 172.21.15.152:0/874659117 shutdown_connections 2024-04-25T01:41:38.134 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.147+0000 7fd307934700 1 -- 172.21.15.152:0/874659117 wait complete. 2024-04-25T01:41:38.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.147+0000 7fd307934700 1 Processor -- start 2024-04-25T01:41:38.136 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.147+0000 7fd307934700 1 -- start start 2024-04-25T01:41:38.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.147+0000 7fd307934700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd300100630 0x7fd300102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:38.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.147+0000 7fd307934700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd300103610 0x7fd3001015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:38.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.147+0000 7fd307934700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd300103fb0 0x7fd300101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:38.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.147+0000 7fd307934700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd30010df50 con 0x7fd300100630 2024-04-25T01:41:38.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.147+0000 7fd307934700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd30010ddd0 con 0x7fd300103fb0 2024-04-25T01:41:38.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.147+0000 7fd307934700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd30010e0d0 con 0x7fd300103610 2024-04-25T01:41:38.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd305ed1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd300103fb0 0x7fd300101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:38.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd305ed1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd300103fb0 0x7fd300101b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41260/0 (socket says 172.21.15.152:41260) 2024-04-25T01:41:38.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd305ed1700 1 -- 172.21.15.152:0/2959181643 learned_addr learned my addr 172.21.15.152:0/2959181643 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-25T01:41:38.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd3056d0700 1 --2- 172.21.15.152:0/2959181643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd300100630 0x7fd300102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:38.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd304ecf700 1 --2- 172.21.15.152:0/2959181643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd300103610 0x7fd3001015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:38.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd305ed1700 1 -- 172.21.15.152:0/2959181643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd300103610 msgr2=0x7fd3001015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:38.137 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd305ed1700 1 --2- 172.21.15.152:0/2959181643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd300103610 0x7fd3001015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:38.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd305ed1700 1 -- 172.21.15.152:0/2959181643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd300100630 msgr2=0x7fd300102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:38.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd305ed1700 1 --2- 172.21.15.152:0/2959181643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd300100630 0x7fd300102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:38.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd305ed1700 1 -- 172.21.15.152:0/2959181643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3001023b0 con 0x7fd300103fb0 2024-04-25T01:41:38.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd305ed1700 1 --2- 172.21.15.152:0/2959181643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd300103fb0 0x7fd300101b00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800ada0 tx=0x7fd2f800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:38.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd2f67fc700 1 -- 172.21.15.152:0/2959181643 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8017020 con 0x7fd300103fb0 2024-04-25T01:41:38.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd2f67fc700 1 -- 172.21.15.152:0/2959181643 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2f800a0d0 con 0x7fd300103fb0 2024-04-25T01:41:38.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd2f67fc700 1 -- 172.21.15.152:0/2959181643 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8005480 con 0x7fd300103fb0 2024-04-25T01:41:38.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd307934700 1 -- 172.21.15.152:0/2959181643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3001a2630 con 0x7fd300103fb0 2024-04-25T01:41:38.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.148+0000 7fd307934700 1 -- 172.21.15.152:0/2959181643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3001a2af0 con 0x7fd300103fb0 2024-04-25T01:41:38.138 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.149+0000 7fd307934700 1 -- 172.21.15.152:0/2959181643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd300068f20 con 0x7fd300103fb0 2024-04-25T01:41:38.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.150+0000 7fd2f67fc700 1 -- 172.21.15.152:0/2959181643 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd2f8005620 con 0x7fd300103fb0 2024-04-25T01:41:38.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.151+0000 7fd2f67fc700 1 --2- 172.21.15.152:0/2959181643 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd2ec0a2ca0 0x7fd2ec0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:38.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.151+0000 7fd2f67fc700 1 -- 172.21.15.152:0/2959181643 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fd2f80ce4f0 con 0x7fd300103fb0 2024-04-25T01:41:38.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.151+0000 7fd3056d0700 1 --2- 172.21.15.152:0/2959181643 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd2ec0a2ca0 0x7fd2ec0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:38.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.151+0000 7fd3056d0700 1 --2- 172.21.15.152:0/2959181643 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd2ec0a2ca0 0x7fd2ec0a5160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc003890 tx=0x7fd2fc005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:38.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.155+0000 7fd2f67fc700 1 -- 172.21.15.152:0/2959181643 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd2f8021020 con 0x7fd300103fb0 2024-04-25T01:41:38.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:38.360+0000 7fd307934700 1 -- 172.21.15.152:0/2959181643 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd30002dfa0 con 0x7fd2ec0a2ca0 2024-04-25T01:41:39.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:38 smithi078 ceph-mon[29469]: Adjusting osd_memory_target on smithi078 to 4272M 2024-04-25T01:41:39.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:38 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:41:39.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:38 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:41:39.003 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:38 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:39.044 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:38 smithi152 ceph-mon[32465]: Adjusting osd_memory_target on smithi078 to 4272M 2024-04-25T01:41:39.044 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:38 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:41:39.044 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:38 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:41:39.044 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:38 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:39.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:38 smithi196 ceph-mon[32978]: Adjusting osd_memory_target on smithi078 to 4272M 2024-04-25T01:41:39.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:38 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:41:39.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:38 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:41:39.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:38 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:40.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:39 smithi078 ceph-mon[29469]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:40.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:39 smithi078 ceph-mon[29469]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:40.005 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:39 smithi152 ceph-mon[32465]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:40.005 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:39 smithi152 ceph-mon[32465]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:40.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:39 smithi196 ceph-mon[32978]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:40.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:39 smithi196 ceph-mon[32978]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:42.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:41 smithi078 ceph-mon[29469]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:42.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:41 smithi078 ceph-mon[29469]: from='client.? 172.21.15.152:0/2663940086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1"}]: dispatch 2024-04-25T01:41:42.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:41 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1"}]: dispatch 2024-04-25T01:41:42.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:41 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1"}]': finished 2024-04-25T01:41:42.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:41 smithi078 ceph-mon[29469]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T01:41:42.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:41:42.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:41 smithi078 ceph-mon[29469]: from='client.? 172.21.15.152:0/3851144964' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:41:42.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:41 smithi152 ceph-mon[32465]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:42.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:41 smithi152 ceph-mon[32465]: from='client.? 172.21.15.152:0/2663940086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1"}]: dispatch 2024-04-25T01:41:42.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:41 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1"}]: dispatch 2024-04-25T01:41:42.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:41 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1"}]': finished 2024-04-25T01:41:42.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:41 smithi152 ceph-mon[32465]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T01:41:42.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:41 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:41:42.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:41 smithi152 ceph-mon[32465]: from='client.? 172.21.15.152:0/3851144964' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:41:42.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:41 smithi196 ceph-mon[32978]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:42.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:41 smithi196 ceph-mon[32978]: from='client.? 172.21.15.152:0/2663940086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1"}]: dispatch 2024-04-25T01:41:42.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:41 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1"}]: dispatch 2024-04-25T01:41:42.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:41 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1"}]': finished 2024-04-25T01:41:42.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:41 smithi196 ceph-mon[32978]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T01:41:42.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:41 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:41:42.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:41 smithi196 ceph-mon[32978]: from='client.? 172.21.15.152:0/3851144964' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:41:44.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:43 smithi078 ceph-mon[29469]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:44.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:43 smithi152 ceph-mon[32465]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:44.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:43 smithi196 ceph-mon[32978]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:46.178 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:45 smithi152 ceph-mon[32465]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:46.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:45 smithi078 ceph-mon[29469]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:46.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:45 smithi196 ceph-mon[32978]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:48.174 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:47 smithi152 ceph-mon[32465]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:48.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:47 smithi078 ceph-mon[29469]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:48.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:47 smithi196 ceph-mon[32978]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:49.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:48 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T01:41:49.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:48 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:49.254 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:48 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T01:41:49.254 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:48 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:49.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:48 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T01:41:49.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:48 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:50.183 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:49 smithi152 ceph-mon[32465]: Deploying daemon osd.10 on smithi152 2024-04-25T01:41:50.183 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:49 smithi152 ceph-mon[32465]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:50.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:49 smithi078 ceph-mon[29469]: Deploying daemon osd.10 on smithi152 2024-04-25T01:41:50.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:49 smithi078 ceph-mon[29469]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:50.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:49 smithi196 ceph-mon[32978]: Deploying daemon osd.10 on smithi152 2024-04-25T01:41:50.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:49 smithi196 ceph-mon[32978]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:52.248 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:51 smithi152 ceph-mon[32465]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:52.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:51 smithi078 ceph-mon[29469]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:52.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:51 smithi196 ceph-mon[32978]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:54.190 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:53 smithi152 ceph-mon[32465]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:54.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:53 smithi078 ceph-mon[29469]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:54.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:53 smithi196 ceph-mon[32978]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:54.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:54.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:54.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:54.987 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:54.987 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:54.987 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:55.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:55.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:55.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:56.246 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:55 smithi152 ceph-mon[32465]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:56.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:55 smithi078 ceph-mon[29469]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:56.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:55 smithi196 ceph-mon[32978]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:58.201 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:57 smithi152 ceph-mon[32465]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:58.201 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:57 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:58.201 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:57 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:58.201 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:57 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:58.201 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:57 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:58.201 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:57 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:58.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:57 smithi078 ceph-mon[29469]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:58.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:57 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:58.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:57 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:58.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:57 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:58.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:57 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:58.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:57 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:58.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:58.278+0000 7fd2f67fc700 1 -- 172.21.15.152:0/2959181643 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd30002dfa0 con 0x7fd2ec0a2ca0 2024-04-25T01:41:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 10 on host 'smithi152' 2024-04-25T01:41:58.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:58.280+0000 7fd307934700 1 -- 172.21.15.152:0/2959181643 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd2ec0a2ca0 msgr2=0x7fd2ec0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:58.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:58.280+0000 7fd307934700 1 --2- 172.21.15.152:0/2959181643 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd2ec0a2ca0 0x7fd2ec0a5160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc003890 tx=0x7fd2fc005fd0 comp rx=0 tx=0).stop 2024-04-25T01:41:58.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:58.280+0000 7fd307934700 1 -- 172.21.15.152:0/2959181643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd300103fb0 msgr2=0x7fd300101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:58.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:58.280+0000 7fd307934700 1 --2- 172.21.15.152:0/2959181643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd300103fb0 0x7fd300101b00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800ada0 tx=0x7fd2f800c330 comp rx=0 tx=0).stop 2024-04-25T01:41:58.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:58.280+0000 7fd307934700 1 -- 172.21.15.152:0/2959181643 shutdown_connections 2024-04-25T01:41:58.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:58.280+0000 7fd307934700 1 --2- 172.21.15.152:0/2959181643 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd2ec0a2ca0 0x7fd2ec0a5160 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:58.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:58.280+0000 7fd307934700 1 --2- 172.21.15.152:0/2959181643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd300103fb0 0x7fd300101b00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:58.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:58.280+0000 7fd307934700 1 --2- 172.21.15.152:0/2959181643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd300100630 0x7fd300102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:58.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:58.280+0000 7fd307934700 1 --2- 172.21.15.152:0/2959181643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd300103610 0x7fd3001015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:58.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:58.280+0000 7fd307934700 1 -- 172.21.15.152:0/2959181643 >> 172.21.15.152:0/2959181643 conn(0x7fd3000fbf90 msgr2=0x7fd3001068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:58.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:58.281+0000 7fd307934700 1 -- 172.21.15.152:0/2959181643 shutdown_connections 2024-04-25T01:41:58.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T01:41:58.281+0000 7fd307934700 1 -- 172.21.15.152:0/2959181643 wait complete. 2024-04-25T01:41:58.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:57 smithi196 ceph-mon[32978]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:41:58.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:57 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:58.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:57 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:58.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:57 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:58.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:57 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:58.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:57 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:58.891 DEBUG:teuthology.orchestra.run.smithi152:osd.10> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T01:41:58.893 INFO:tasks.cephadm:Deploying osd.11 on smithi196 with /dev/vg_nvme/lv_1... 2024-04-25T01:41:58.893 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T01:41:59.142 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:41:59.217 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:58 smithi152 ceph-mon[32465]: from='osd.10 [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T01:41:59.217 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:58 smithi152 ceph-mon[32465]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T01:41:59.217 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:58 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:59.217 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:58 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:59.217 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:58 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:59.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:58 smithi078 ceph-mon[29469]: from='osd.10 [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T01:41:59.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:58 smithi078 ceph-mon[29469]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T01:41:59.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:58 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:59.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:58 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:59.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:58 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:59.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:58 smithi196 ceph-mon[32978]: from='osd.10 [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T01:41:59.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:58 smithi196 ceph-mon[32978]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T01:41:59.263 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:58 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:59.264 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:58 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:41:59.264 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:58 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:00.045 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:59 smithi152 ceph-mon[32465]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:42:00.045 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:59 smithi152 ceph-mon[32465]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T01:42:00.045 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:59 smithi152 ceph-mon[32465]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T01:42:00.045 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:59 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:00.045 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:59 smithi152 ceph-mon[32465]: from='osd.10 [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:42:00.045 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:41:59 smithi152 ceph-mon[32465]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:42:00.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:59 smithi078 ceph-mon[29469]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:42:00.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:59 smithi078 ceph-mon[29469]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T01:42:00.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:59 smithi078 ceph-mon[29469]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T01:42:00.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:59 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:00.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:59 smithi078 ceph-mon[29469]: from='osd.10 [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:42:00.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:41:59 smithi078 ceph-mon[29469]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:42:00.305 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:59 smithi196 ceph-mon[32978]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:42:00.305 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:59 smithi196 ceph-mon[32978]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T01:42:00.305 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:59 smithi196 ceph-mon[32978]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T01:42:00.305 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:59 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:00.305 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:59 smithi196 ceph-mon[32978]: from='osd.10 [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:42:00.305 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:41:59 smithi196 ceph-mon[32978]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T01:42:00.795 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:42:00 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:42:00.493+0000 7f4227709700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T01:42:00.795 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:42:00 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:42:00.505+0000 7f421f4f4700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:42:01.234 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:00 smithi152 ceph-mon[32465]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T01:42:01.234 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:00 smithi152 ceph-mon[32465]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T01:42:01.234 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:00 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:01.234 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:00 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:01.234 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:00 smithi152 ceph-mon[32465]: from='osd.10 ' entity='osd.10' 2024-04-25T01:42:01.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:00 smithi078 ceph-mon[29469]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T01:42:01.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:00 smithi078 ceph-mon[29469]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T01:42:01.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:00 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:01.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:00 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:01.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:00 smithi078 ceph-mon[29469]: from='osd.10 ' entity='osd.10' 2024-04-25T01:42:01.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:00 smithi196 ceph-mon[32978]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T01:42:01.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:00 smithi196 ceph-mon[32978]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T01:42:01.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:00 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:01.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:00 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:01.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:00 smithi196 ceph-mon[32978]: from='osd.10 ' entity='osd.10' 2024-04-25T01:42:01.384 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T01:42:01.385 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T01:42:01.385 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-04-25T01:42:01.385 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-04-25T01:42:01.385 INFO:teuthology.orchestra.run.smithi196.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0138886 s, 755 MB/s 2024-04-25T01:42:01.385 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-04-25T01:42:02.092 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_1 2024-04-25T01:42:02.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:01 smithi078 ceph-mon[29469]: purged_snaps scrub starts 2024-04-25T01:42:02.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:01 smithi078 ceph-mon[29469]: purged_snaps scrub ok 2024-04-25T01:42:02.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:01 smithi078 ceph-mon[29469]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:42:02.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:01 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:02.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:01 smithi078 ceph-mon[29469]: osd.10 [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476] boot 2024-04-25T01:42:02.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:01 smithi078 ceph-mon[29469]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T01:42:02.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:01 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:02.294 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:01 smithi152 ceph-mon[32465]: purged_snaps scrub starts 2024-04-25T01:42:02.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:01 smithi152 ceph-mon[32465]: purged_snaps scrub ok 2024-04-25T01:42:02.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:01 smithi152 ceph-mon[32465]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:42:02.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:01 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:02.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:01 smithi152 ceph-mon[32465]: osd.10 [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476] boot 2024-04-25T01:42:02.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:01 smithi152 ceph-mon[32465]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T01:42:02.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:01 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:02.333 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:42:02.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:01 smithi196 ceph-mon[32978]: purged_snaps scrub starts 2024-04-25T01:42:02.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:01 smithi196 ceph-mon[32978]: purged_snaps scrub ok 2024-04-25T01:42:02.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:01 smithi196 ceph-mon[32978]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T01:42:02.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:01 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:02.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:01 smithi196 ceph-mon[32978]: osd.10 [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476] boot 2024-04-25T01:42:02.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:01 smithi196 ceph-mon[32978]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T01:42:02.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:01 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T01:42:03.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:02 smithi196 ceph-mon[32978]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T01:42:03.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:03.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:03.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:03.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:03.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:03.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:03.114 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:03.114 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:03.114 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:03.114 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:02 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:03.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:02 smithi078 ceph-mon[29469]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T01:42:03.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:03.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:03.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:03.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:03.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:03.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:03.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:03.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:03.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:03.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:02 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:03.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:02 smithi152 ceph-mon[32465]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T01:42:03.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:03.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:03.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:03.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:03.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:03.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:03.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:03.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:03.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:03.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:02 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:04.055 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:03 smithi196 ceph-mon[32978]: Detected new or changed devices on smithi152 2024-04-25T01:42:04.055 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:03 smithi196 ceph-mon[32978]: Adjusting osd_memory_target on smithi152 to 4272M 2024-04-25T01:42:04.055 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:03 smithi196 ceph-mon[32978]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:04.055 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:03 smithi196 ceph-mon[32978]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T01:42:04.157 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.169+0000 7f2a0e61a700 1 -- 172.21.15.196:0/1330098316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a081056d0 msgr2=0x7f2a08107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.169+0000 7f2a0e61a700 1 --2- 172.21.15.196:0/1330098316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a081056d0 0x7f2a08107ac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009990 tx=0x7f29fc009ca0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.169+0000 7f2a0e61a700 1 -- 172.21.15.196:0/1330098316 shutdown_connections 2024-04-25T01:42:04.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.169+0000 7f2a0e61a700 1 --2- 172.21.15.196:0/1330098316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a08108000 0x7f2a0810a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.169+0000 7f2a0e61a700 1 --2- 172.21.15.196:0/1330098316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a08102da0 0x7f2a08105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.169+0000 7f2a0e61a700 1 --2- 172.21.15.196:0/1330098316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a081056d0 0x7f2a08107ac0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.169+0000 7f2a0e61a700 1 -- 172.21.15.196:0/1330098316 >> 172.21.15.196:0/1330098316 conn(0x7f2a080fc750 msgr2=0x7f2a080feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:04.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.169+0000 7f2a0e61a700 1 -- 172.21.15.196:0/1330098316 shutdown_connections 2024-04-25T01:42:04.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.169+0000 7f2a0e61a700 1 -- 172.21.15.196:0/1330098316 wait complete. 2024-04-25T01:42:04.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.170+0000 7f2a0e61a700 1 Processor -- start 2024-04-25T01:42:04.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.170+0000 7f2a0e61a700 1 -- start start 2024-04-25T01:42:04.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.170+0000 7f2a0e61a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a08102da0 0x7f2a081984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a0e61a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a081056d0 0x7f2a08198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a0e61a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a08108000 0x7f2a0819fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a0e61a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0810df00 con 0x7f2a08102da0 2024-04-25T01:42:04.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a0e61a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0810dd80 con 0x7f2a08108000 2024-04-25T01:42:04.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a0e61a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0810e080 con 0x7f2a081056d0 2024-04-25T01:42:04.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a077fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a081056d0 0x7f2a08198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a077fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a081056d0 0x7f2a08198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:47674/0 (socket says 172.21.15.196:47674) 2024-04-25T01:42:04.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a077fe700 1 -- 172.21.15.196:0/2268167719 learned_addr learned my addr 172.21.15.196:0/2268167719 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T01:42:04.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a07fff700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a08102da0 0x7f2a081984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a0cbb7700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a08108000 0x7f2a0819fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a077fe700 1 -- 172.21.15.196:0/2268167719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a08108000 msgr2=0x7f2a0819fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a077fe700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a08108000 0x7f2a0819fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a077fe700 1 -- 172.21.15.196:0/2268167719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a08102da0 msgr2=0x7f2a081984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a077fe700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a08102da0 0x7f2a081984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a077fe700 1 -- 172.21.15.196:0/2268167719 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a081a0120 con 0x7f2a081056d0 2024-04-25T01:42:04.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a07fff700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a08102da0 0x7f2a081984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:42:04.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a077fe700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a081056d0 0x7f2a08198a00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f29fc004790 tx=0x7f29fc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:04.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a057fa700 1 -- 172.21.15.196:0/2268167719 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc027070 con 0x7f2a081056d0 2024-04-25T01:42:04.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a057fa700 1 -- 172.21.15.196:0/2268167719 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29fc004210 con 0x7f2a081056d0 2024-04-25T01:42:04.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a0e61a700 1 -- 172.21.15.196:0/2268167719 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a081a03b0 con 0x7f2a081056d0 2024-04-25T01:42:04.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a057fa700 1 -- 172.21.15.196:0/2268167719 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc00f460 con 0x7f2a081056d0 2024-04-25T01:42:04.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.171+0000 7f2a0e61a700 1 -- 172.21.15.196:0/2268167719 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a081a08e0 con 0x7f2a081056d0 2024-04-25T01:42:04.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.172+0000 7f2a0e61a700 1 -- 172.21.15.196:0/2268167719 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a08068f20 con 0x7f2a081056d0 2024-04-25T01:42:04.163 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.173+0000 7f2a057fa700 1 -- 172.21.15.196:0/2268167719 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f29fc02a070 con 0x7f2a081056d0 2024-04-25T01:42:04.163 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.174+0000 7f2a057fa700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29f00a2ca0 0x7f29f00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.163 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.174+0000 7f2a057fa700 1 -- 172.21.15.196:0/2268167719 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f29fc0cfc00 con 0x7f2a081056d0 2024-04-25T01:42:04.163 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.174+0000 7f2a07fff700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29f00a2ca0 0x7f29f00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.163 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.175+0000 7f2a07fff700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29f00a2ca0 0x7f29f00a5160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f29f8005950 tx=0x7f29f80058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:04.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.178+0000 7f2a057fa700 1 -- 172.21.15.196:0/2268167719 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f29fc013070 con 0x7f2a081056d0 2024-04-25T01:42:04.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:03 smithi078 ceph-mon[29469]: Detected new or changed devices on smithi152 2024-04-25T01:42:04.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:03 smithi078 ceph-mon[29469]: Adjusting osd_memory_target on smithi152 to 4272M 2024-04-25T01:42:04.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:03 smithi078 ceph-mon[29469]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:04.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:03 smithi078 ceph-mon[29469]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T01:42:04.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:03 smithi152 ceph-mon[32465]: Detected new or changed devices on smithi152 2024-04-25T01:42:04.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:03 smithi152 ceph-mon[32465]: Adjusting osd_memory_target on smithi152 to 4272M 2024-04-25T01:42:04.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:03 smithi152 ceph-mon[32465]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:04.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:03 smithi152 ceph-mon[32465]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T01:42:04.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:04.381+0000 7f2a0e61a700 1 -- 172.21.15.196:0/2268167719 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2a08062670 con 0x7f29f00a2ca0 2024-04-25T01:42:05.249 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:04 smithi196 ceph-mon[32978]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T01:42:05.249 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:04 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:42:05.249 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:04 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:42:05.249 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:04 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:05.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:04 smithi078 ceph-mon[29469]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T01:42:05.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:04 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:42:05.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:04 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:42:05.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:04 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:05.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:04 smithi152 ceph-mon[32465]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T01:42:05.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:04 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:42:05.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:04 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:42:05.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:04 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:06.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:05 smithi078 ceph-mon[29469]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:06.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:05 smithi078 ceph-mon[29469]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:06.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:05 smithi078 ceph-mon[29469]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T01:42:06.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:06 smithi152 ceph-mon[32465]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:06.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:06 smithi152 ceph-mon[32465]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:06.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:06 smithi152 ceph-mon[32465]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T01:42:06.312 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:06 smithi196 ceph-mon[32978]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:06.312 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:06 smithi196 ceph-mon[32978]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:06.312 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:06 smithi196 ceph-mon[32978]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T01:42:08.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:08 smithi078 ceph-mon[29469]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:08.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:08 smithi078 ceph-mon[29469]: from='client.? 172.21.15.196:0/2308569620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a29b5606-d549-4980-bb67-adaf71ac22bf"}]: dispatch 2024-04-25T01:42:08.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:08 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a29b5606-d549-4980-bb67-adaf71ac22bf"}]: dispatch 2024-04-25T01:42:08.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:08 smithi078 ceph-mon[29469]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a29b5606-d549-4980-bb67-adaf71ac22bf"}]': finished 2024-04-25T01:42:08.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:08 smithi078 ceph-mon[29469]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T01:42:08.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:08 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:08.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:08 smithi078 ceph-mon[29469]: from='client.? 172.21.15.196:0/2168081852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:42:08.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:08 smithi152 ceph-mon[32465]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:08.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:08 smithi152 ceph-mon[32465]: from='client.? 172.21.15.196:0/2308569620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a29b5606-d549-4980-bb67-adaf71ac22bf"}]: dispatch 2024-04-25T01:42:08.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:08 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a29b5606-d549-4980-bb67-adaf71ac22bf"}]: dispatch 2024-04-25T01:42:08.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:08 smithi152 ceph-mon[32465]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a29b5606-d549-4980-bb67-adaf71ac22bf"}]': finished 2024-04-25T01:42:08.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:08 smithi152 ceph-mon[32465]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T01:42:08.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:08 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:08.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:08 smithi152 ceph-mon[32465]: from='client.? 172.21.15.196:0/2168081852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:42:08.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:08 smithi196 ceph-mon[32978]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:08.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:08 smithi196 ceph-mon[32978]: from='client.? 172.21.15.196:0/2308569620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a29b5606-d549-4980-bb67-adaf71ac22bf"}]: dispatch 2024-04-25T01:42:08.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:08 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a29b5606-d549-4980-bb67-adaf71ac22bf"}]: dispatch 2024-04-25T01:42:08.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:08 smithi196 ceph-mon[32978]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a29b5606-d549-4980-bb67-adaf71ac22bf"}]': finished 2024-04-25T01:42:08.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:08 smithi196 ceph-mon[32978]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T01:42:08.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:08 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:08.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:08 smithi196 ceph-mon[32978]: from='client.? 172.21.15.196:0/2168081852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:42:10.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:10 smithi078 ceph-mon[29469]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T01:42:10.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:10 smithi152 ceph-mon[32465]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T01:42:10.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:10 smithi196 ceph-mon[32978]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T01:42:12.112 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:12 smithi196 ceph-mon[32978]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T01:42:12.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:12 smithi078 ceph-mon[29469]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T01:42:12.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:12 smithi152 ceph-mon[32465]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T01:42:14.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:14 smithi078 ceph-mon[29469]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T01:42:14.280 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:14 smithi196 ceph-mon[32978]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T01:42:14.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:14 smithi152 ceph-mon[32465]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T01:42:15.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T01:42:15.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:15 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:15.362 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T01:42:15.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:15 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:15.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T01:42:15.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:15 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:16.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:16 smithi196 ceph-mon[32978]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:42:16.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:16 smithi196 ceph-mon[32978]: Deploying daemon osd.11 on smithi196 2024-04-25T01:42:16.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:16 smithi152 ceph-mon[32465]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:42:16.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:16 smithi152 ceph-mon[32465]: Deploying daemon osd.11 on smithi196 2024-04-25T01:42:16.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:16 smithi078 ceph-mon[29469]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:42:16.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:16 smithi078 ceph-mon[29469]: Deploying daemon osd.11 on smithi196 2024-04-25T01:42:18.261 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:18 smithi196 ceph-mon[32978]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:42:18.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:18 smithi152 ceph-mon[32465]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:42:18.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:18 smithi078 ceph-mon[29469]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T01:42:20.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:20 smithi152 ceph-mon[32465]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T01:42:20.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:20 smithi196 ceph-mon[32978]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T01:42:20.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:20 smithi078 ceph-mon[29469]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T01:42:22.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:22 smithi196 ceph-mon[32978]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:22.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:22 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:22.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:22 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:22.113 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:22 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:22.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:22 smithi152 ceph-mon[32465]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:22.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:22 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:22.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:22 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:22.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:22 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:22.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:22 smithi078 ceph-mon[29469]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:22.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:22 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:22.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:22 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:22.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:22 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:24.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:24 smithi152 ceph-mon[32465]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:24.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:24 smithi196 ceph-mon[32978]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:24.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:24 smithi078 ceph-mon[29469]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:25.294 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 11 on host 'smithi196' 2024-04-25T01:42:25.294 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:25.305+0000 7f2a057fa700 1 -- 172.21.15.196:0/2268167719 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2a08062670 con 0x7f29f00a2ca0 2024-04-25T01:42:25.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:25.308+0000 7f2a0e61a700 1 -- 172.21.15.196:0/2268167719 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29f00a2ca0 msgr2=0x7f29f00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:25.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:25.308+0000 7f2a0e61a700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29f00a2ca0 0x7f29f00a5160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f29f8005950 tx=0x7f29f80058e0 comp rx=0 tx=0).stop 2024-04-25T01:42:25.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:25.308+0000 7f2a0e61a700 1 -- 172.21.15.196:0/2268167719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a081056d0 msgr2=0x7f2a08198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:25.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:25.308+0000 7f2a0e61a700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a081056d0 0x7f2a08198a00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f29fc004790 tx=0x7f29fc009580 comp rx=0 tx=0).stop 2024-04-25T01:42:25.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:25.308+0000 7f2a0e61a700 1 -- 172.21.15.196:0/2268167719 shutdown_connections 2024-04-25T01:42:25.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:25.308+0000 7f2a0e61a700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29f00a2ca0 0x7f29f00a5160 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:25.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:25.308+0000 7f2a0e61a700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2a08108000 0x7f2a0819fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:25.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:25.308+0000 7f2a0e61a700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a08102da0 0x7f2a081984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:25.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:25.308+0000 7f2a0e61a700 1 --2- 172.21.15.196:0/2268167719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a081056d0 0x7f2a08198a00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:25.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:25.308+0000 7f2a0e61a700 1 -- 172.21.15.196:0/2268167719 >> 172.21.15.196:0/2268167719 conn(0x7f2a080fc750 msgr2=0x7f2a081036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:25.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:25.308+0000 7f2a0e61a700 1 -- 172.21.15.196:0/2268167719 shutdown_connections 2024-04-25T01:42:25.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:25.308+0000 7f2a0e61a700 1 -- 172.21.15.196:0/2268167719 wait complete. 2024-04-25T01:42:25.613 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.613 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.613 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:25.613 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:25.613 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.613 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:25 smithi196 ceph-mon[32978]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:25.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:25 smithi196 ceph-mon[32978]: from='osd.11 [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T01:42:25.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:25 smithi196 ceph-mon[32978]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T01:42:25.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:25 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:25.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.754 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.754 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:25.754 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:25.754 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.754 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:25 smithi078 ceph-mon[29469]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:25.754 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:25 smithi078 ceph-mon[29469]: from='osd.11 [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T01:42:25.754 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:25 smithi078 ceph-mon[29469]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T01:42:25.754 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.754 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.754 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:25 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:25.795 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.795 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.795 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:25.795 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:25.795 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:25 smithi152 ceph-mon[32465]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:25.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:25 smithi152 ceph-mon[32465]: from='osd.11 [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T01:42:25.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:25 smithi152 ceph-mon[32465]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T01:42:25.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:25.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:25 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:26.020 DEBUG:teuthology.orchestra.run.smithi196:osd.11> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T01:42:26.022 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T01:42:26.022 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T01:42:26.273 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:42:26.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:26 smithi196 ceph-mon[32978]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T01:42:26.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:26 smithi196 ceph-mon[32978]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T01:42:26.565 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:26 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:26.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:26 smithi196 ceph-mon[32978]: from='osd.11 [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:42:26.566 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:26 smithi196 ceph-mon[32978]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:42:26.566 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:42:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:42:26.423+0000 7fe12923e700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T01:42:26.566 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:42:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:42:26.438+0000 7fe125031700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:42:26.726 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:26 smithi078 ceph-mon[29469]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T01:42:26.726 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:26 smithi078 ceph-mon[29469]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T01:42:26.726 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:26 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:26.726 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:26 smithi078 ceph-mon[29469]: from='osd.11 [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:42:26.726 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:26 smithi078 ceph-mon[29469]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:42:26.795 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:26 smithi152 ceph-mon[32465]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T01:42:26.795 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:26 smithi152 ceph-mon[32465]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T01:42:26.795 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:26 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:26.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:26 smithi152 ceph-mon[32465]: from='osd.11 [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:42:26.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:26 smithi152 ceph-mon[32465]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-25T01:42:27.582 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:27 smithi196 ceph-mon[32978]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-25T01:42:27.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:27 smithi196 ceph-mon[32978]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T01:42:27.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:27 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:27.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:27 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:27.583 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:27 smithi196 ceph-mon[32978]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:27.681 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:27 smithi078 ceph-mon[29469]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-25T01:42:27.681 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:27 smithi078 ceph-mon[29469]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T01:42:27.681 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:27 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:27.681 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:27 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:27.681 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:27 smithi078 ceph-mon[29469]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:27.795 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:27 smithi152 ceph-mon[32465]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-25T01:42:27.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:27 smithi152 ceph-mon[32465]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T01:42:27.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:27 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:27.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:27 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:27.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:27 smithi152 ceph-mon[32465]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T01:42:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.057+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2303595511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53bc103490 msgr2=0x7f53bc1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.057+0000 7f53c0dc3700 1 --2- 172.21.15.78:0/2303595511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53bc103490 0x7f53bc1038f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009920 tx=0x7f53a4009c30 comp rx=0 tx=0).stop 2024-04-25T01:42:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.057+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2303595511 shutdown_connections 2024-04-25T01:42:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.057+0000 7f53c0dc3700 1 --2- 172.21.15.78:0/2303595511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53bc103e30 0x7f53bc10a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.057+0000 7f53c0dc3700 1 --2- 172.21.15.78:0/2303595511 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53bc102290 0x7f53bc1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.057+0000 7f53c0dc3700 1 --2- 172.21.15.78:0/2303595511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53bc103490 0x7f53bc1038f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.057+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2303595511 >> 172.21.15.78:0/2303595511 conn(0x7f53bc0fd830 msgr2=0x7f53bc0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.057+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2303595511 shutdown_connections 2024-04-25T01:42:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.057+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2303595511 wait complete. 2024-04-25T01:42:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53c0dc3700 1 Processor -- start 2024-04-25T01:42:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53c0dc3700 1 -- start start 2024-04-25T01:42:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53c0dc3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53bc102290 0x7f53bc198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53c0dc3700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53bc103490 0x7f53bc198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53c0dc3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53bc103e30 0x7f53bc19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53c0dc3700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc10de70 con 0x7f53bc103490 2024-04-25T01:42:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53c0dc3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc10dcf0 con 0x7f53bc102290 2024-04-25T01:42:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53c0dc3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc10dff0 con 0x7f53bc103e30 2024-04-25T01:42:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53b1d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53bc103490 0x7f53bc198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53b1d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53bc103490 0x7f53bc198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35562/0 (socket says 172.21.15.78:35562) 2024-04-25T01:42:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53b1d9b700 1 -- 172.21.15.78:0/2210557911 learned_addr learned my addr 172.21.15.78:0/2210557911 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53bad9d700 1 --2- 172.21.15.78:0/2210557911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53bc103e30 0x7f53bc19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53ba59c700 1 --2- 172.21.15.78:0/2210557911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53bc102290 0x7f53bc198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53b1d9b700 1 -- 172.21.15.78:0/2210557911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53bc103e30 msgr2=0x7f53bc19fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53b1d9b700 1 --2- 172.21.15.78:0/2210557911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53bc103e30 0x7f53bc19fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53b1d9b700 1 -- 172.21.15.78:0/2210557911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53bc102290 msgr2=0x7f53bc198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53b1d9b700 1 --2- 172.21.15.78:0/2210557911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53bc102290 0x7f53bc198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.058+0000 7f53b1d9b700 1 -- 172.21.15.78:0/2210557911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53bc1a01a0 con 0x7f53bc103490 2024-04-25T01:42:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.059+0000 7f53b1d9b700 1 --2- 172.21.15.78:0/2210557911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53bc103490 0x7f53bc198a80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f53a400fca0 tx=0x7f53a4005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.059+0000 7f53b3fff700 1 -- 172.21.15.78:0/2210557911 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4025070 con 0x7f53bc103490 2024-04-25T01:42:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.059+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2210557911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53bc1a0430 con 0x7f53bc103490 2024-04-25T01:42:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.059+0000 7f53b3fff700 1 -- 172.21.15.78:0/2210557911 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53a4005650 con 0x7f53bc103490 2024-04-25T01:42:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.059+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2210557911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53bc1a08e0 con 0x7f53bc103490 2024-04-25T01:42:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.059+0000 7f53b3fff700 1 -- 172.21.15.78:0/2210557911 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4003cb0 con 0x7f53bc103490 2024-04-25T01:42:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.060+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2210557911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53bc068f20 con 0x7f53bc103490 2024-04-25T01:42:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.061+0000 7f53b3fff700 1 -- 172.21.15.78:0/2210557911 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f53a4028070 con 0x7f53bc103490 2024-04-25T01:42:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.062+0000 7f53b3fff700 1 --2- 172.21.15.78:0/2210557911 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f53a80a2c20 0x7f53a80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.062+0000 7f53ba59c700 1 --2- 172.21.15.78:0/2210557911 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f53a80a2c20 0x7f53a80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.062+0000 7f53b3fff700 1 -- 172.21.15.78:0/2210557911 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f53a40cf8a0 con 0x7f53bc103490 2024-04-25T01:42:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.062+0000 7f53ba59c700 1 --2- 172.21.15.78:0/2210557911 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f53a80a2c20 0x7f53a80a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009550 tx=0x7f53ac0091b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.065+0000 7f53b3fff700 1 -- 172.21.15.78:0/2210557911 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f53a402f020 con 0x7f53bc103490 2024-04-25T01:42:28.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.258+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2210557911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f53bc02db30 con 0x7f53bc103490 2024-04-25T01:42:28.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.258+0000 7f53b3fff700 1 -- 172.21.15.78:0/2210557911 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f53a4013030 con 0x7f53bc103490 2024-04-25T01:42:28.248 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:42:28.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.261+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2210557911 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f53a80a2c20 msgr2=0x7f53a80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:28.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.261+0000 7f53c0dc3700 1 --2- 172.21.15.78:0/2210557911 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f53a80a2c20 0x7f53a80a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009550 tx=0x7f53ac0091b0 comp rx=0 tx=0).stop 2024-04-25T01:42:28.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.261+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2210557911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53bc103490 msgr2=0x7f53bc198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:28.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.261+0000 7f53c0dc3700 1 --2- 172.21.15.78:0/2210557911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53bc103490 0x7f53bc198a80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f53a400fca0 tx=0x7f53a4005250 comp rx=0 tx=0).stop 2024-04-25T01:42:28.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.261+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2210557911 shutdown_connections 2024-04-25T01:42:28.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.261+0000 7f53c0dc3700 1 --2- 172.21.15.78:0/2210557911 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f53a80a2c20 0x7f53a80a50e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:28.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.261+0000 7f53c0dc3700 1 --2- 172.21.15.78:0/2210557911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53bc103e30 0x7f53bc19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:28.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.261+0000 7f53c0dc3700 1 --2- 172.21.15.78:0/2210557911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53bc102290 0x7f53bc198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:28.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.261+0000 7f53c0dc3700 1 --2- 172.21.15.78:0/2210557911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53bc103490 0x7f53bc198a80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:28.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.261+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2210557911 >> 172.21.15.78:0/2210557911 conn(0x7f53bc0fd830 msgr2=0x7f53bc1068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:28.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.261+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2210557911 shutdown_connections 2024-04-25T01:42:28.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:28.261+0000 7f53c0dc3700 1 -- 172.21.15.78:0/2210557911 wait complete. 2024-04-25T01:42:28.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:28 smithi078 ceph-mon[29469]: purged_snaps scrub starts 2024-04-25T01:42:28.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:28 smithi078 ceph-mon[29469]: purged_snaps scrub ok 2024-04-25T01:42:28.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:28 smithi078 ceph-mon[29469]: osd.11 [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986] boot 2024-04-25T01:42:28.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:28 smithi078 ceph-mon[29469]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T01:42:28.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:28 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:28.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:28 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2210557911' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T01:42:28.697 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:28 smithi152 ceph-mon[32465]: purged_snaps scrub starts 2024-04-25T01:42:28.697 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:28 smithi152 ceph-mon[32465]: purged_snaps scrub ok 2024-04-25T01:42:28.697 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:28 smithi152 ceph-mon[32465]: osd.11 [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986] boot 2024-04-25T01:42:28.697 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:28 smithi152 ceph-mon[32465]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T01:42:28.697 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:28 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:28.697 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:28 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2210557911' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T01:42:28.765 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:28 smithi196 ceph-mon[32978]: purged_snaps scrub starts 2024-04-25T01:42:28.765 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:28 smithi196 ceph-mon[32978]: purged_snaps scrub ok 2024-04-25T01:42:28.765 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:28 smithi196 ceph-mon[32978]: osd.11 [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986] boot 2024-04-25T01:42:28.765 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:28 smithi196 ceph-mon[32978]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T01:42:28.765 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:28 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T01:42:28.765 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:28 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2210557911' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T01:42:28.926 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714009347,"num_in_osds":12,"osd_in_since":1714009327,"num_remapped_pgs":0} 2024-04-25T01:42:28.926 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T01:42:29.170 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:42:29.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:29 smithi078 ceph-mon[29469]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T01:42:29.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:29 smithi078 ceph-mon[29469]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:29.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:29.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:29.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:29.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:29.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:29.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:29.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:29.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:29.561 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:29.562 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:29 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:29.613 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:29 smithi196 ceph-mon[32978]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T01:42:29.613 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:29 smithi196 ceph-mon[32978]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:29.613 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:29.613 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:29.613 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:29.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:29.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:29.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:29.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:29.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:29.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:29.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:29 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:29.648 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:29 smithi152 ceph-mon[32465]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T01:42:29.649 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:29 smithi152 ceph-mon[32465]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:29.649 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:29.649 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:29.649 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:29.649 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:29.649 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:29.649 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:42:29.649 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:29.649 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:29.649 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:29.649 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:29 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:30.746 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:30 smithi196 ceph-mon[32978]: Detected new or changed devices on smithi196 2024-04-25T01:42:30.747 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:30 smithi196 ceph-mon[32978]: Adjusting osd_memory_target on smithi196 to 4272M 2024-04-25T01:42:30.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:30 smithi078 ceph-mon[29469]: Detected new or changed devices on smithi196 2024-04-25T01:42:30.754 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:30 smithi078 ceph-mon[29469]: Adjusting osd_memory_target on smithi196 to 4272M 2024-04-25T01:42:30.795 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:30 smithi152 ceph-mon[32465]: Detected new or changed devices on smithi196 2024-04-25T01:42:30.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:30 smithi152 ceph-mon[32465]: Adjusting osd_memory_target on smithi196 to 4272M 2024-04-25T01:42:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.004+0000 7fbfd2730700 1 -- 172.21.15.78:0/3735704914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfcc1064f0 msgr2=0x7fbfcc10e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.004+0000 7fbfd2730700 1 --2- 172.21.15.78:0/3735704914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfcc1064f0 0x7fbfcc10e0c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400b0d0 tx=0x7fbfc400b3e0 comp rx=0 tx=0).stop 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.004+0000 7fbfd2730700 1 -- 172.21.15.78:0/3735704914 shutdown_connections 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.004+0000 7fbfd2730700 1 --2- 172.21.15.78:0/3735704914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfcc1064f0 0x7fbfcc10e0c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.004+0000 7fbfd2730700 1 --2- 172.21.15.78:0/3735704914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc101690 0x7fbfcc101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.004+0000 7fbfd2730700 1 --2- 172.21.15.78:0/3735704914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc102040 0x7fbfcc105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.004+0000 7fbfd2730700 1 -- 172.21.15.78:0/3735704914 >> 172.21.15.78:0/3735704914 conn(0x7fbfcc077f30 msgr2=0x7fbfcc078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.004+0000 7fbfd2730700 1 -- 172.21.15.78:0/3735704914 shutdown_connections 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.004+0000 7fbfd2730700 1 -- 172.21.15.78:0/3735704914 wait complete. 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.005+0000 7fbfd2730700 1 Processor -- start 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.005+0000 7fbfd2730700 1 -- start start 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.005+0000 7fbfd2730700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc101690 0x7fbfcc19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.005+0000 7fbfd2730700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc102040 0x7fbfcc19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.005+0000 7fbfd2730700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfcc1064f0 0x7fbfcc198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.005+0000 7fbfd2730700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc111500 con 0x7fbfcc1064f0 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.005+0000 7fbfd2730700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc111380 con 0x7fbfcc102040 2024-04-25T01:42:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.005+0000 7fbfd2730700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc111680 con 0x7fbfcc101690 2024-04-25T01:42:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.005+0000 7fbfd0ccd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfcc1064f0 0x7fbfcc198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.005+0000 7fbfd0ccd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfcc1064f0 0x7fbfcc198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44590/0 (socket says 172.21.15.78:44590) 2024-04-25T01:42:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.005+0000 7fbfd0ccd700 1 -- 172.21.15.78:0/2461268820 learned_addr learned my addr 172.21.15.78:0/2461268820 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.005+0000 7fbfcbfff700 1 --2- 172.21.15.78:0/2461268820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc101690 0x7fbfcc19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.005+0000 7fbfcb7fe700 1 --2- 172.21.15.78:0/2461268820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc102040 0x7fbfcc19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.006+0000 7fbfd0ccd700 1 -- 172.21.15.78:0/2461268820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc101690 msgr2=0x7fbfcc19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.006+0000 7fbfd0ccd700 1 --2- 172.21.15.78:0/2461268820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc101690 0x7fbfcc19e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.006+0000 7fbfd0ccd700 1 -- 172.21.15.78:0/2461268820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc102040 msgr2=0x7fbfcc19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.006+0000 7fbfd0ccd700 1 --2- 172.21.15.78:0/2461268820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc102040 0x7fbfcc19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.006+0000 7fbfd0ccd700 1 -- 172.21.15.78:0/2461268820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfcc199060 con 0x7fbfcc1064f0 2024-04-25T01:42:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.006+0000 7fbfd0ccd700 1 --2- 172.21.15.78:0/2461268820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfcc1064f0 0x7fbfcc198900 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4006070 tx=0x7fbfc4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.006+0000 7fbfc97fa700 1 -- 172.21.15.78:0/2461268820 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4021070 con 0x7fbfcc1064f0 2024-04-25T01:42:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.006+0000 7fbfd2730700 1 -- 172.21.15.78:0/2461268820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfcc1992f0 con 0x7fbfcc1064f0 2024-04-25T01:42:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.006+0000 7fbfc97fa700 1 -- 172.21.15.78:0/2461268820 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc4007b80 con 0x7fbfcc1064f0 2024-04-25T01:42:30.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.006+0000 7fbfc97fa700 1 -- 172.21.15.78:0/2461268820 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc40052a0 con 0x7fbfcc1064f0 2024-04-25T01:42:30.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.006+0000 7fbfd2730700 1 -- 172.21.15.78:0/2461268820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfcc1a5610 con 0x7fbfcc1064f0 2024-04-25T01:42:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.007+0000 7fbfd2730700 1 -- 172.21.15.78:0/2461268820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfcc068f20 con 0x7fbfcc1064f0 2024-04-25T01:42:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.008+0000 7fbfc97fa700 1 -- 172.21.15.78:0/2461268820 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbfc4024070 con 0x7fbfcc1064f0 2024-04-25T01:42:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.008+0000 7fbfc97fa700 1 --2- 172.21.15.78:0/2461268820 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbfb40a2c20 0x7fbfb40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.009+0000 7fbfcbfff700 1 --2- 172.21.15.78:0/2461268820 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbfb40a2c20 0x7fbfb40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.009+0000 7fbfc97fa700 1 -- 172.21.15.78:0/2461268820 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbfc400f020 con 0x7fbfcc1064f0 2024-04-25T01:42:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.009+0000 7fbfcbfff700 1 --2- 172.21.15.78:0/2461268820 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbfb40a2c20 0x7fbfb40a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc199a40 tx=0x7fbfbc00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.012+0000 7fbfc97fa700 1 -- 172.21.15.78:0/2461268820 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbfc402b020 con 0x7fbfcc1064f0 2024-04-25T01:42:31.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.206+0000 7fbfd2730700 1 -- 172.21.15.78:0/2461268820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbfcc02db30 con 0x7fbfcc1064f0 2024-04-25T01:42:31.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.207+0000 7fbfc97fa700 1 -- 172.21.15.78:0/2461268820 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18479 (secure 0 0 0) 0x7fbfc4097e20 con 0x7fbfcc1064f0 2024-04-25T01:42:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:42:31.197 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":82,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:42:28.429191+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:42:31.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.209+0000 7fbfd2730700 1 -- 172.21.15.78:0/2461268820 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbfb40a2c20 msgr2=0x7fbfb40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:31.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.209+0000 7fbfd2730700 1 --2- 172.21.15.78:0/2461268820 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbfb40a2c20 0x7fbfb40a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc199a40 tx=0x7fbfbc00b410 comp rx=0 tx=0).stop 2024-04-25T01:42:31.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.209+0000 7fbfd2730700 1 -- 172.21.15.78:0/2461268820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfcc1064f0 msgr2=0x7fbfcc198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:31.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.209+0000 7fbfd2730700 1 --2- 172.21.15.78:0/2461268820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfcc1064f0 0x7fbfcc198900 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4006070 tx=0x7fbfc4004d00 comp rx=0 tx=0).stop 2024-04-25T01:42:31.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.210+0000 7fbfd2730700 1 -- 172.21.15.78:0/2461268820 shutdown_connections 2024-04-25T01:42:31.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.210+0000 7fbfd2730700 1 --2- 172.21.15.78:0/2461268820 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbfb40a2c20 0x7fbfb40a50e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:31.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.210+0000 7fbfd2730700 1 --2- 172.21.15.78:0/2461268820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfcc101690 0x7fbfcc19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:31.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.210+0000 7fbfd2730700 1 --2- 172.21.15.78:0/2461268820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbfcc1064f0 0x7fbfcc198900 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:31.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.210+0000 7fbfd2730700 1 --2- 172.21.15.78:0/2461268820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfcc102040 0x7fbfcc19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:31.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.210+0000 7fbfd2730700 1 -- 172.21.15.78:0/2461268820 >> 172.21.15.78:0/2461268820 conn(0x7fbfcc077f30 msgr2=0x7fbfcc06ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:31.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.210+0000 7fbfd2730700 1 -- 172.21.15.78:0/2461268820 shutdown_connections 2024-04-25T01:42:31.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:31.210+0000 7fbfd2730700 1 -- 172.21.15.78:0/2461268820 wait complete. 2024-04-25T01:42:31.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:31 smithi078 ceph-mon[29469]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:31.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:31 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2461268820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:42:31.795 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:31 smithi152 ceph-mon[32465]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:31.795 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:31 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2461268820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:42:31.835 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:38:39.136418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:42:31.835 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T01:42:31.863 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:31 smithi196 ceph-mon[32978]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:31.863 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:31 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2461268820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:42:32.072 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:42:33.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.777+0000 7f75ad6f8700 1 -- 172.21.15.78:0/1380710181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75a8102550 msgr2=0x7f75a81029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.777+0000 7f75ad6f8700 1 --2- 172.21.15.78:0/1380710181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75a8102550 0x7f75a81029d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f759c009990 tx=0x7f759c009ca0 comp rx=0 tx=0).stop 2024-04-25T01:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.777+0000 7f75ad6f8700 1 -- 172.21.15.78:0/1380710181 shutdown_connections 2024-04-25T01:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.777+0000 7f75ad6f8700 1 --2- 172.21.15.78:0/1380710181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75a8102f10 0x7f75a810f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.777+0000 7f75ad6f8700 1 --2- 172.21.15.78:0/1380710181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a81083f0 0x7f75a81087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.777+0000 7f75ad6f8700 1 --2- 172.21.15.78:0/1380710181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75a8102550 0x7f75a81029d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.777+0000 7f75ad6f8700 1 -- 172.21.15.78:0/1380710181 >> 172.21.15.78:0/1380710181 conn(0x7f75a807a1c0 msgr2=0x7f75a807a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.777+0000 7f75ad6f8700 1 -- 172.21.15.78:0/1380710181 shutdown_connections 2024-04-25T01:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.778+0000 7f75ad6f8700 1 -- 172.21.15.78:0/1380710181 wait complete. 2024-04-25T01:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.778+0000 7f75ad6f8700 1 Processor -- start 2024-04-25T01:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.778+0000 7f75ad6f8700 1 -- start start 2024-04-25T01:42:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.778+0000 7f75ad6f8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75a8102550 0x7f75a819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.778+0000 7f75ad6f8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75a8102f10 0x7f75a819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.778+0000 7f75ad6f8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a81083f0 0x7f75a8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.778+0000 7f75ad6f8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f75a8112610 con 0x7f75a8102550 2024-04-25T01:42:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.778+0000 7f75ad6f8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f75a8112490 con 0x7f75a81083f0 2024-04-25T01:42:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.778+0000 7f75ad6f8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75a8112790 con 0x7f75a8102f10 2024-04-25T01:42:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f75a6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75a8102550 0x7f75a819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f75a6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75a8102550 0x7f75a819e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44606/0 (socket says 172.21.15.78:44606) 2024-04-25T01:42:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f75a6ffd700 1 -- 172.21.15.78:0/3423074192 learned_addr learned my addr 172.21.15.78:0/3423074192 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f75a67fc700 1 --2- 172.21.15.78:0/3423074192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75a8102f10 0x7f75a819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f75a6ffd700 1 -- 172.21.15.78:0/3423074192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75a8102f10 msgr2=0x7f75a819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f75a6ffd700 1 --2- 172.21.15.78:0/3423074192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75a8102f10 0x7f75a819ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f75a6ffd700 1 -- 172.21.15.78:0/3423074192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a81083f0 msgr2=0x7f75a8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:33.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f75a6ffd700 1 --2- 172.21.15.78:0/3423074192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a81083f0 0x7f75a8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:33.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f75a6ffd700 1 -- 172.21.15.78:0/3423074192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a81990f0 con 0x7f75a8102550 2024-04-25T01:42:33.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f75a6ffd700 1 --2- 172.21.15.78:0/3423074192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75a8102550 0x7f75a819e7d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f759800d950 tx=0x7f759800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:33.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f758ffff700 1 -- 172.21.15.78:0/3423074192 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598016070 con 0x7f75a8102550 2024-04-25T01:42:33.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f75ad6f8700 1 -- 172.21.15.78:0/3423074192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a81993e0 con 0x7f75a8102550 2024-04-25T01:42:33.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f758ffff700 1 -- 172.21.15.78:0/3423074192 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7598009b40 con 0x7f75a8102550 2024-04-25T01:42:33.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f758ffff700 1 -- 172.21.15.78:0/3423074192 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759800f620 con 0x7f75a8102550 2024-04-25T01:42:33.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.779+0000 7f75ad6f8700 1 -- 172.21.15.78:0/3423074192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a81a56a0 con 0x7f75a8102550 2024-04-25T01:42:33.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.780+0000 7f75ad6f8700 1 -- 172.21.15.78:0/3423074192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a8068f20 con 0x7f75a8102550 2024-04-25T01:42:33.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.781+0000 7f758ffff700 1 -- 172.21.15.78:0/3423074192 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7598019070 con 0x7f75a8102550 2024-04-25T01:42:33.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.782+0000 7f758ffff700 1 --2- 172.21.15.78:0/3423074192 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f75900a2c20 0x7f75900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:33.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.782+0000 7f75a67fc700 1 --2- 172.21.15.78:0/3423074192 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f75900a2c20 0x7f75900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:33.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.782+0000 7f758ffff700 1 -- 172.21.15.78:0/3423074192 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f75980cebd0 con 0x7f75a8102550 2024-04-25T01:42:33.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.782+0000 7f75a67fc700 1 --2- 172.21.15.78:0/3423074192 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f75900a2c20 0x7f75900a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f759c005cd0 tx=0x7f759c005c40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:33.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.786+0000 7f758ffff700 1 -- 172.21.15.78:0/3423074192 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f759809bf30 con 0x7f75a8102550 2024-04-25T01:42:33.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.984+0000 7f75ad6f8700 1 -- 172.21.15.78:0/3423074192 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75a802db30 con 0x7f75a8102550 2024-04-25T01:42:33.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.984+0000 7f758ffff700 1 -- 172.21.15.78:0/3423074192 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f759809b830 con 0x7f75a8102550 2024-04-25T01:42:33.973 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T01:42:33.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.986+0000 7f75ad6f8700 1 -- 172.21.15.78:0/3423074192 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f75900a2c20 msgr2=0x7f75900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:33.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.986+0000 7f75ad6f8700 1 --2- 172.21.15.78:0/3423074192 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f75900a2c20 0x7f75900a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f759c005cd0 tx=0x7f759c005c40 comp rx=0 tx=0).stop 2024-04-25T01:42:33.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.986+0000 7f75ad6f8700 1 -- 172.21.15.78:0/3423074192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75a8102550 msgr2=0x7f75a819e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:33.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.986+0000 7f75ad6f8700 1 --2- 172.21.15.78:0/3423074192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75a8102550 0x7f75a819e7d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f759800d950 tx=0x7f759800dd10 comp rx=0 tx=0).stop 2024-04-25T01:42:33.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.986+0000 7f75ad6f8700 1 -- 172.21.15.78:0/3423074192 shutdown_connections 2024-04-25T01:42:33.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.986+0000 7f75ad6f8700 1 --2- 172.21.15.78:0/3423074192 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f75900a2c20 0x7f75900a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:33.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.986+0000 7f75ad6f8700 1 --2- 172.21.15.78:0/3423074192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a81083f0 0x7f75a8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:33.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.986+0000 7f75ad6f8700 1 --2- 172.21.15.78:0/3423074192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75a8102550 0x7f75a819e7d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:33.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.986+0000 7f75ad6f8700 1 --2- 172.21.15.78:0/3423074192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75a8102f10 0x7f75a819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:33.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.986+0000 7f75ad6f8700 1 -- 172.21.15.78:0/3423074192 >> 172.21.15.78:0/3423074192 conn(0x7f75a807a1c0 msgr2=0x7f75a8101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:33.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.986+0000 7f75ad6f8700 1 -- 172.21.15.78:0/3423074192 shutdown_connections 2024-04-25T01:42:33.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:33.986+0000 7f75ad6f8700 1 -- 172.21.15.78:0/3423074192 wait complete. 2024-04-25T01:42:34.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:33 smithi078 ceph-mon[29469]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:34.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:33 smithi152 ceph-mon[32465]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:34.363 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:33 smithi196 ceph-mon[32978]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:34.687 INFO:tasks.cephadm:Adding mds.a on smithi078 2024-04-25T01:42:34.688 INFO:tasks.cephadm:Adding mds.d on smithi078 2024-04-25T01:42:34.688 INFO:tasks.cephadm:Adding mds.g on smithi078 2024-04-25T01:42:34.688 INFO:tasks.cephadm:Adding mds.j on smithi078 2024-04-25T01:42:34.688 INFO:tasks.cephadm:Adding mds.b on smithi152 2024-04-25T01:42:34.688 INFO:tasks.cephadm:Adding mds.e on smithi152 2024-04-25T01:42:34.688 INFO:tasks.cephadm:Adding mds.h on smithi152 2024-04-25T01:42:34.688 INFO:tasks.cephadm:Adding mds.k on smithi152 2024-04-25T01:42:34.688 INFO:tasks.cephadm:Adding mds.c on smithi196 2024-04-25T01:42:34.688 INFO:tasks.cephadm:Adding mds.f on smithi196 2024-04-25T01:42:34.688 INFO:tasks.cephadm:Adding mds.i on smithi196 2024-04-25T01:42:34.688 INFO:tasks.cephadm:Adding mds.l on smithi196 2024-04-25T01:42:34.689 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l' 2024-04-25T01:42:34.937 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:42:35.059 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:34 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3423074192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:42:35.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:34 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3423074192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:42:35.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:34 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3423074192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:42:36.077 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:35 smithi196 ceph-mon[32978]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:36.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:35 smithi078 ceph-mon[29469]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:36.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:35 smithi152 ceph-mon[32465]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:36.606 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.614+0000 7efd74970700 1 -- 172.21.15.196:0/3944713340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd70074c90 msgr2=0x7efd70075110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.606 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.614+0000 7efd74970700 1 --2- 172.21.15.196:0/3944713340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd70074c90 0x7efd70075110 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efd64009990 tx=0x7efd64009ca0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.614+0000 7efd74970700 1 -- 172.21.15.196:0/3944713340 shutdown_connections 2024-04-25T01:42:36.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.614+0000 7efd74970700 1 --2- 172.21.15.196:0/3944713340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd7010a150 0x7efd7010c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.614+0000 7efd74970700 1 --2- 172.21.15.196:0/3944713340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd70076260 0x7efd700746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.614+0000 7efd74970700 1 --2- 172.21.15.196:0/3944713340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd70074c90 0x7efd70075110 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.614+0000 7efd74970700 1 -- 172.21.15.196:0/3944713340 >> 172.21.15.196:0/3944713340 conn(0x7efd700fd820 msgr2=0x7efd700ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:36.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.614+0000 7efd74970700 1 -- 172.21.15.196:0/3944713340 shutdown_connections 2024-04-25T01:42:36.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.614+0000 7efd74970700 1 -- 172.21.15.196:0/3944713340 wait complete. 2024-04-25T01:42:36.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.614+0000 7efd74970700 1 Processor -- start 2024-04-25T01:42:36.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd74970700 1 -- start start 2024-04-25T01:42:36.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd74970700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd70074c90 0x7efd7019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:36.607 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd74970700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd70076260 0x7efd7019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:36.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd74970700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd7010a150 0x7efd701a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:36.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd74970700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efd70110080 con 0x7efd70076260 2024-04-25T01:42:36.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd74970700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efd7010ff00 con 0x7efd70074c90 2024-04-25T01:42:36.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd74970700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd70110200 con 0x7efd7010a150 2024-04-25T01:42:36.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd6ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd7010a150 0x7efd701a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:36.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd6ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd7010a150 0x7efd701a1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:53760/0 (socket says 172.21.15.196:53760) 2024-04-25T01:42:36.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd6ed9d700 1 -- 172.21.15.196:0/303920808 learned_addr learned my addr 172.21.15.196:0/303920808 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T01:42:36.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd6e59c700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd70074c90 0x7efd7019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:36.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd6dd9b700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd70076260 0x7efd7019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:36.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd6ed9d700 1 -- 172.21.15.196:0/303920808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd70074c90 msgr2=0x7efd7019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd6ed9d700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd70074c90 0x7efd7019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd6ed9d700 1 -- 172.21.15.196:0/303920808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd70076260 msgr2=0x7efd7019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd6ed9d700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd70076260 0x7efd7019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd6ed9d700 1 -- 172.21.15.196:0/303920808 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd701a2350 con 0x7efd7010a150 2024-04-25T01:42:36.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd6dd9b700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd70076260 0x7efd7019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:42:36.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd6ed9d700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd7010a150 0x7efd701a1cb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efd68007f10 tx=0x7efd6800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:36.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd5f7fe700 1 -- 172.21.15.196:0/303920808 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68004bb0 con 0x7efd7010a150 2024-04-25T01:42:36.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.615+0000 7efd74970700 1 -- 172.21.15.196:0/303920808 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd701a2640 con 0x7efd7010a150 2024-04-25T01:42:36.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.616+0000 7efd5f7fe700 1 -- 172.21.15.196:0/303920808 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd68004d50 con 0x7efd7010a150 2024-04-25T01:42:36.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.616+0000 7efd5f7fe700 1 -- 172.21.15.196:0/303920808 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68005a60 con 0x7efd7010a150 2024-04-25T01:42:36.609 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.616+0000 7efd74970700 1 -- 172.21.15.196:0/303920808 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd701a2b80 con 0x7efd7010a150 2024-04-25T01:42:36.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.617+0000 7efd74970700 1 -- 172.21.15.196:0/303920808 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd70068f20 con 0x7efd7010a150 2024-04-25T01:42:36.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.618+0000 7efd5f7fe700 1 -- 172.21.15.196:0/303920808 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efd6801a020 con 0x7efd7010a150 2024-04-25T01:42:36.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.618+0000 7efd5f7fe700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efd580a2c20 0x7efd580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:36.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.618+0000 7efd5f7fe700 1 -- 172.21.15.196:0/303920808 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efd680ce3b0 con 0x7efd7010a150 2024-04-25T01:42:36.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.618+0000 7efd6e59c700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efd580a2c20 0x7efd580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:36.611 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.619+0000 7efd6e59c700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efd580a2c20 0x7efd580a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efd70076640 tx=0x7efd60009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:36.612 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.623+0000 7efd5f7fe700 1 -- 172.21.15.196:0/303920808 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efd68010070 con 0x7efd7010a150 2024-04-25T01:42:36.819 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.829+0000 7efd74970700 1 -- 172.21.15.196:0/303920808 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}) -- 0x7efd70062830 con 0x7efd580a2c20 2024-04-25T01:42:36.826 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.837+0000 7efd5f7fe700 1 -- 172.21.15.196:0/303920808 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7efd70062830 con 0x7efd580a2c20 2024-04-25T01:42:36.827 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mds.all update... 2024-04-25T01:42:36.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.839+0000 7efd74970700 1 -- 172.21.15.196:0/303920808 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efd580a2c20 msgr2=0x7efd580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.839+0000 7efd74970700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efd580a2c20 0x7efd580a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efd70076640 tx=0x7efd60009260 comp rx=0 tx=0).stop 2024-04-25T01:42:36.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.839+0000 7efd74970700 1 -- 172.21.15.196:0/303920808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd7010a150 msgr2=0x7efd701a1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.839+0000 7efd74970700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd7010a150 0x7efd701a1cb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efd68007f10 tx=0x7efd6800d3b0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.839+0000 7efd74970700 1 -- 172.21.15.196:0/303920808 shutdown_connections 2024-04-25T01:42:36.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.839+0000 7efd74970700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efd580a2c20 0x7efd580a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.839+0000 7efd74970700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd7010a150 0x7efd701a1cb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.839+0000 7efd74970700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd70074c90 0x7efd7019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.839+0000 7efd74970700 1 --2- 172.21.15.196:0/303920808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efd70076260 0x7efd7019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.839+0000 7efd74970700 1 -- 172.21.15.196:0/303920808 >> 172.21.15.196:0/303920808 conn(0x7efd700fd820 msgr2=0x7efd70106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:36.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.839+0000 7efd74970700 1 -- 172.21.15.196:0/303920808 shutdown_connections 2024-04-25T01:42:36.830 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T01:42:36.839+0000 7efd74970700 1 -- 172.21.15.196:0/303920808 wait complete. 2024-04-25T01:42:37.480 DEBUG:teuthology.orchestra.run.smithi078:mds.a> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T01:42:37.483 DEBUG:teuthology.orchestra.run.smithi078:mds.d> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T01:42:37.526 DEBUG:teuthology.orchestra.run.smithi078:mds.g> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T01:42:37.569 DEBUG:teuthology.orchestra.run.smithi078:mds.j> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T01:42:37.612 DEBUG:teuthology.orchestra.run.smithi152:mds.b> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T01:42:37.614 DEBUG:teuthology.orchestra.run.smithi152:mds.e> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T01:42:37.657 DEBUG:teuthology.orchestra.run.smithi152:mds.h> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T01:42:37.700 DEBUG:teuthology.orchestra.run.smithi152:mds.k> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T01:42:37.744 DEBUG:teuthology.orchestra.run.smithi196:mds.c> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T01:42:37.746 DEBUG:teuthology.orchestra.run.smithi196:mds.f> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T01:42:37.789 DEBUG:teuthology.orchestra.run.smithi196:mds.i> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T01:42:37.832 DEBUG:teuthology.orchestra.run.smithi196:mds.l> sudo journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T01:42:37.875 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T01:42:37.875 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:42:37.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.992+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/1605520582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb78101870 msgr2=0x7ffb78101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:37.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.992+0000 7ffb7e3b6640 1 --2- 172.21.15.78:0/1605520582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb78101870 0x7ffb78101cf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009900 tx=0x7ffb6802a410 comp rx=0 tx=0).stop 2024-04-25T01:42:37.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.992+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/1605520582 shutdown_connections 2024-04-25T01:42:37.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.992+0000 7ffb7e3b6640 1 --2- 172.21.15.78:0/1605520582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb78102230 0x7ffb78108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.992+0000 7ffb7e3b6640 1 --2- 172.21.15.78:0/1605520582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb78101870 0x7ffb78101cf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.992+0000 7ffb7e3b6640 1 --2- 172.21.15.78:0/1605520582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb78100680 0x7ffb78100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.992+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/1605520582 >> 172.21.15.78:0/1605520582 conn(0x7ffb780fbbf0 msgr2=0x7ffb780fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:37.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.992+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/1605520582 shutdown_connections 2024-04-25T01:42:37.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.992+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/1605520582 wait complete. 2024-04-25T01:42:37.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.992+0000 7ffb7e3b6640 1 Processor -- start 2024-04-25T01:42:37.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.992+0000 7ffb7e3b6640 1 -- start start 2024-04-25T01:42:37.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb7e3b6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb78100680 0x7ffb78197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:37.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb7e3b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb78101870 0x7ffb78197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:37.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb7e3b6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb78102230 0x7ffb7819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:37.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb7e3b6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7810c300 con 0x7ffb78101870 2024-04-25T01:42:37.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb7e3b6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7810c180 con 0x7ffb78100680 2024-04-25T01:42:37.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb7e3b6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7810c480 con 0x7ffb78102230 2024-04-25T01:42:37.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb777fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb78101870 0x7ffb78197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:37.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb777fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb78101870 0x7ffb78197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44620/0 (socket says 172.21.15.78:44620) 2024-04-25T01:42:37.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb777fe640 1 -- 172.21.15.78:0/2928373156 learned_addr learned my addr 172.21.15.78:0/2928373156 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:37.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb7c953640 1 --2- 172.21.15.78:0/2928373156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb78102230 0x7ffb7819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:37.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb77fff640 1 --2- 172.21.15.78:0/2928373156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb78100680 0x7ffb78197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:37.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb777fe640 1 -- 172.21.15.78:0/2928373156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb78102230 msgr2=0x7ffb7819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:37.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb777fe640 1 --2- 172.21.15.78:0/2928373156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb78102230 0x7ffb7819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb777fe640 1 -- 172.21.15.78:0/2928373156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb78100680 msgr2=0x7ffb78197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:37.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb777fe640 1 --2- 172.21.15.78:0/2928373156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb78100680 0x7ffb78197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb777fe640 1 -- 172.21.15.78:0/2928373156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7819f6c0 con 0x7ffb78101870 2024-04-25T01:42:37.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb777fe640 1 --2- 172.21.15.78:0/2928373156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb78101870 0x7ffb78197fe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009230 tx=0x7ffb68004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:37.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb757fa640 1 -- 172.21.15.78:0/2928373156 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6803f030 con 0x7ffb78101870 2024-04-25T01:42:37.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb757fa640 1 -- 172.21.15.78:0/2928373156 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb68003ce0 con 0x7ffb78101870 2024-04-25T01:42:37.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/2928373156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7819f950 con 0x7ffb78101870 2024-04-25T01:42:37.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb757fa640 1 -- 172.21.15.78:0/2928373156 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68035070 con 0x7ffb78101870 2024-04-25T01:42:37.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.993+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/2928373156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7819fe40 con 0x7ffb78101870 2024-04-25T01:42:37.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.995+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/2928373156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb781a0010 con 0x7ffb78101870 2024-04-25T01:42:37.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.996+0000 7ffb757fa640 1 -- 172.21.15.78:0/2928373156 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffb68042030 con 0x7ffb78101870 2024-04-25T01:42:37.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.996+0000 7ffb757fa640 1 --2- 172.21.15.78:0/2928373156 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ffb640a2b90 0x7ffb640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:37.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.996+0000 7ffb77fff640 1 --2- 172.21.15.78:0/2928373156 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ffb640a2b90 0x7ffb640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:37.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.996+0000 7ffb757fa640 1 -- 172.21.15.78:0/2928373156 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ffb680ef590 con 0x7ffb78101870 2024-04-25T01:42:37.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:37.997+0000 7ffb77fff640 1 --2- 172.21.15.78:0/2928373156 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ffb640a2b90 0x7ffb640a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb60005fd0 tx=0x7ffb60005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:37.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.001+0000 7ffb757fa640 1 -- 172.21.15.78:0/2928373156 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffb680bc990 con 0x7ffb78101870 2024-04-25T01:42:38.046 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:37 smithi152 ceph-mon[32465]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:38.046 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:37 smithi152 ceph-mon[32465]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-04-25T01:42:38.046 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:38.046 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:38.046 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:37 smithi152 ceph-mon[32465]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:38.046 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:38.046 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:38.046 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:38.047 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:38.047 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:38.047 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:37 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:38.086 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:37 smithi196 ceph-mon[32978]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:38.086 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:37 smithi196 ceph-mon[32978]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-04-25T01:42:38.086 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:38.086 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:38.086 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:37 smithi196 ceph-mon[32978]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:38.086 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:38.086 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:38.086 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:38.086 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:38.086 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:38.086 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:37 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:38.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.177+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/2928373156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb781a03a0 con 0x7ffb78101870 2024-04-25T01:42:38.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.178+0000 7ffb757fa640 1 -- 172.21.15.78:0/2928373156 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18479 (secure 0 0 0) 0x7ffb680bc290 con 0x7ffb78101870 2024-04-25T01:42:38.168 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:42:38.168 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":82,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:42:28.429191+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:42:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.180+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/2928373156 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ffb640a2b90 msgr2=0x7ffb640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.180+0000 7ffb7e3b6640 1 --2- 172.21.15.78:0/2928373156 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ffb640a2b90 0x7ffb640a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb60005fd0 tx=0x7ffb60005850 comp rx=0 tx=0).stop 2024-04-25T01:42:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.180+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/2928373156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb78101870 msgr2=0x7ffb78197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.180+0000 7ffb7e3b6640 1 --2- 172.21.15.78:0/2928373156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb78101870 0x7ffb78197fe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009230 tx=0x7ffb68004360 comp rx=0 tx=0).stop 2024-04-25T01:42:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.181+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/2928373156 shutdown_connections 2024-04-25T01:42:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.181+0000 7ffb7e3b6640 1 --2- 172.21.15.78:0/2928373156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb78102230 0x7ffb7819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.181+0000 7ffb7e3b6640 1 --2- 172.21.15.78:0/2928373156 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ffb640a2b90 0x7ffb640a5050 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.181+0000 7ffb7e3b6640 1 --2- 172.21.15.78:0/2928373156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb78101870 0x7ffb78197fe0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.181+0000 7ffb7e3b6640 1 --2- 172.21.15.78:0/2928373156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb78100680 0x7ffb78197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.181+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/2928373156 >> 172.21.15.78:0/2928373156 conn(0x7ffb780fbbf0 msgr2=0x7ffb780fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.181+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/2928373156 shutdown_connections 2024-04-25T01:42:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.181+0000 7ffb7e3b6640 1 -- 172.21.15.78:0/2928373156 wait complete. 2024-04-25T01:42:38.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:37 smithi078 ceph-mon[29469]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:38.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:37 smithi078 ceph-mon[29469]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-04-25T01:42:38.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:38.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:38.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:37 smithi078 ceph-mon[29469]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:38.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:38.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:38.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:38.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:38.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:38.181 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:37 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:38.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:38:39.136418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:42:38.182 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:42:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 -- 172.21.15.78:0/1262661126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26c0102970 msgr2=0x7f26c0102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 --2- 172.21.15.78:0/1262661126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26c0102970 0x7f26c0102df0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009900 tx=0x7f26a802a410 comp rx=0 tx=0).stop 2024-04-25T01:42:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 -- 172.21.15.78:0/1262661126 shutdown_connections 2024-04-25T01:42:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 --2- 172.21.15.78:0/1262661126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26c0103330 0x7f26c0109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 --2- 172.21.15.78:0/1262661126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26c0102970 0x7f26c0102df0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 --2- 172.21.15.78:0/1262661126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26c0101780 0x7f26c0101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 -- 172.21.15.78:0/1262661126 >> 172.21.15.78:0/1262661126 conn(0x7f26c00fccf0 msgr2=0x7f26c00ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 -- 172.21.15.78:0/1262661126 shutdown_connections 2024-04-25T01:42:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 -- 172.21.15.78:0/1262661126 wait complete. 2024-04-25T01:42:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 Processor -- start 2024-04-25T01:42:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 -- start start 2024-04-25T01:42:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26c0101780 0x7f26c00754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26c0102970 0x7f26c0073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26c0103330 0x7f26c0074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f26c010d470 con 0x7f26c0103330 2024-04-25T01:42:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f26c010d2f0 con 0x7f26c0102970 2024-04-25T01:42:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.294+0000 7f26c4ad4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f26c010d5f0 con 0x7f26c0101780 2024-04-25T01:42:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26bed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26c0103330 0x7f26c0074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26bed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26c0103330 0x7f26c0074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44632/0 (socket says 172.21.15.78:44632) 2024-04-25T01:42:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26bed9d640 1 -- 172.21.15.78:0/2386937227 learned_addr learned my addr 172.21.15.78:0/2386937227 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26b7fff640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26c0102970 0x7f26c0073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26be59c640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26c0101780 0x7f26c00754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26bed9d640 1 -- 172.21.15.78:0/2386937227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26c0101780 msgr2=0x7f26c00754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26bed9d640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26c0101780 0x7f26c00754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26bed9d640 1 -- 172.21.15.78:0/2386937227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26c0102970 msgr2=0x7f26c0073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26bed9d640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26c0102970 0x7f26c0073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26bed9d640 1 -- 172.21.15.78:0/2386937227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c0074900 con 0x7f26c0103330 2024-04-25T01:42:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26be59c640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26c0101780 0x7f26c00754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:42:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26bed9d640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26c0103330 0x7f26c0074070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009fd0 tx=0x7f26b800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26b77fe640 1 -- 172.21.15.78:0/2386937227 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8017070 con 0x7f26c0103330 2024-04-25T01:42:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26c4ad4640 1 -- 172.21.15.78:0/2386937227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c01a1cb0 con 0x7f26c0103330 2024-04-25T01:42:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26b77fe640 1 -- 172.21.15.78:0/2386937227 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b8003e50 con 0x7f26c0103330 2024-04-25T01:42:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26b77fe640 1 -- 172.21.15.78:0/2386937227 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80106c0 con 0x7f26c0103330 2024-04-25T01:42:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.295+0000 7f26c4ad4640 1 -- 172.21.15.78:0/2386937227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c01a21d0 con 0x7f26c0103330 2024-04-25T01:42:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.296+0000 7f26c4ad4640 1 -- 172.21.15.78:0/2386937227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c0108bc0 con 0x7f26c0103330 2024-04-25T01:42:38.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.297+0000 7f26b77fe640 1 -- 172.21.15.78:0/2386937227 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f26b8003970 con 0x7f26c0103330 2024-04-25T01:42:38.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.298+0000 7f26b77fe640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f26ac0a2b90 0x7f26ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:38.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.298+0000 7f26be59c640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f26ac0a2b90 0x7f26ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:38.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.298+0000 7f26b77fe640 1 -- 172.21.15.78:0/2386937227 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f26b80ce900 con 0x7f26c0103330 2024-04-25T01:42:38.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.298+0000 7f26be59c640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f26ac0a2b90 0x7f26ac0a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f26b0006fd0 tx=0x7f26b0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:38.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.302+0000 7f26b77fe640 1 -- 172.21.15.78:0/2386937227 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f26b809bd00 con 0x7f26c0103330 2024-04-25T01:42:38.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.476+0000 7f26c4ad4640 1 -- 172.21.15.78:0/2386937227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f26c002d320 con 0x7f26c0103330 2024-04-25T01:42:38.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.476+0000 7f26b77fe640 1 -- 172.21.15.78:0/2386937227 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f26b809b600 con 0x7f26c0103330 2024-04-25T01:42:38.465 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T01:42:38.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.478+0000 7f26c4ad4640 1 -- 172.21.15.78:0/2386937227 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f26ac0a2b90 msgr2=0x7f26ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:38.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.478+0000 7f26c4ad4640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f26ac0a2b90 0x7f26ac0a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f26b0006fd0 tx=0x7f26b0008040 comp rx=0 tx=0).stop 2024-04-25T01:42:38.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.478+0000 7f26c4ad4640 1 -- 172.21.15.78:0/2386937227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26c0103330 msgr2=0x7f26c0074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:38.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.478+0000 7f26c4ad4640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26c0103330 0x7f26c0074070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009fd0 tx=0x7f26b800b260 comp rx=0 tx=0).stop 2024-04-25T01:42:38.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.478+0000 7f26c4ad4640 1 -- 172.21.15.78:0/2386937227 shutdown_connections 2024-04-25T01:42:38.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.478+0000 7f26c4ad4640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f26ac0a2b90 0x7f26ac0a5050 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.478+0000 7f26c4ad4640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26c0103330 0x7f26c0074070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.478+0000 7f26c4ad4640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26c0102970 0x7f26c0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.478+0000 7f26c4ad4640 1 --2- 172.21.15.78:0/2386937227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26c0101780 0x7f26c00754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.478+0000 7f26c4ad4640 1 -- 172.21.15.78:0/2386937227 >> 172.21.15.78:0/2386937227 conn(0x7f26c00fccf0 msgr2=0x7f26c0105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:38.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.478+0000 7f26c4ad4640 1 -- 172.21.15.78:0/2386937227 shutdown_connections 2024-04-25T01:42:38.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.478+0000 7f26c4ad4640 1 -- 172.21.15.78:0/2386937227 wait complete. 2024-04-25T01:42:38.479 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:42:38.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.587+0000 7f437de6d640 1 -- 172.21.15.78:0/1988463517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4378102230 msgr2=0x7f4378108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:38.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.587+0000 7f437de6d640 1 --2- 172.21.15.78:0/1988463517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4378102230 0x7f4378108ac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4370009ca0 tx=0x7f437002a8c0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.587+0000 7f437de6d640 1 -- 172.21.15.78:0/1988463517 shutdown_connections 2024-04-25T01:42:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.587+0000 7f437de6d640 1 --2- 172.21.15.78:0/1988463517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4378102230 0x7f4378108ac0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.587+0000 7f437de6d640 1 --2- 172.21.15.78:0/1988463517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4378101870 0x7f4378101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.587+0000 7f437de6d640 1 --2- 172.21.15.78:0/1988463517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4378100680 0x7f4378100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.587+0000 7f437de6d640 1 -- 172.21.15.78:0/1988463517 >> 172.21.15.78:0/1988463517 conn(0x7f43780fbbf0 msgr2=0x7f43780fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.587+0000 7f437de6d640 1 -- 172.21.15.78:0/1988463517 shutdown_connections 2024-04-25T01:42:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.587+0000 7f437de6d640 1 -- 172.21.15.78:0/1988463517 wait complete. 2024-04-25T01:42:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.587+0000 7f437de6d640 1 Processor -- start 2024-04-25T01:42:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.587+0000 7f437de6d640 1 -- start start 2024-04-25T01:42:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.587+0000 7f437de6d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4378100680 0x7f43780694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f437de6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4378101870 0x7f437819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f437de6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4378102230 0x7f437819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f437de6d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f437810c2e0 con 0x7f4378102230 2024-04-25T01:42:38.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f437de6d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f437810c160 con 0x7f4378100680 2024-04-25T01:42:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f437de6d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f437810c460 con 0x7f4378101870 2024-04-25T01:42:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f4377fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4378102230 0x7f437819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f4377fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4378102230 0x7f437819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44654/0 (socket says 172.21.15.78:44654) 2024-04-25T01:42:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f4377fff640 1 -- 172.21.15.78:0/1889357649 learned_addr learned my addr 172.21.15.78:0/1889357649 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f436ffff640 1 --2- 172.21.15.78:0/1889357649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4378101870 0x7f437819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f4377fff640 1 -- 172.21.15.78:0/1889357649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4378101870 msgr2=0x7f437819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f4377fff640 1 --2- 172.21.15.78:0/1889357649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4378101870 0x7f437819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f4377fff640 1 -- 172.21.15.78:0/1889357649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4378100680 msgr2=0x7f43780694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f43777fe640 1 --2- 172.21.15.78:0/1889357649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4378100680 0x7f43780694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f4377fff640 1 --2- 172.21.15.78:0/1889357649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4378100680 0x7f43780694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f4377fff640 1 -- 172.21.15.78:0/1889357649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437819f5c0 con 0x7f4378102230 2024-04-25T01:42:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f4377fff640 1 --2- 172.21.15.78:0/1889357649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4378102230 0x7f437819ed00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4370005e30 tx=0x7f4370031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f43757fa640 1 -- 172.21.15.78:0/1889357649 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370003b70 con 0x7f4378102230 2024-04-25T01:42:38.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f43757fa640 1 -- 172.21.15.78:0/1889357649 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4370003d10 con 0x7f4378102230 2024-04-25T01:42:38.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f437de6d640 1 -- 172.21.15.78:0/1889357649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437819f850 con 0x7f4378102230 2024-04-25T01:42:38.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f43757fa640 1 -- 172.21.15.78:0/1889357649 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437002fd30 con 0x7f4378102230 2024-04-25T01:42:38.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.588+0000 7f437de6d640 1 -- 172.21.15.78:0/1889357649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437819fdf0 con 0x7f4378102230 2024-04-25T01:42:38.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.589+0000 7f437de6d640 1 -- 172.21.15.78:0/1889357649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43781078d0 con 0x7f4378102230 2024-04-25T01:42:38.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.590+0000 7f43757fa640 1 -- 172.21.15.78:0/1889357649 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4370004050 con 0x7f4378102230 2024-04-25T01:42:38.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.591+0000 7f43757fa640 1 --2- 172.21.15.78:0/1889357649 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f43640a2c10 0x7f43640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:38.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.591+0000 7f43777fe640 1 --2- 172.21.15.78:0/1889357649 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f43640a2c10 0x7f43640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:38.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.591+0000 7f43757fa640 1 -- 172.21.15.78:0/1889357649 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f43700f0060 con 0x7f4378102230 2024-04-25T01:42:38.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.591+0000 7f43777fe640 1 --2- 172.21.15.78:0/1889357649 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f43640a2c10 0x7f43640a50d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4368005fd0 tx=0x7f4368005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:38.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.595+0000 7f43757fa640 1 -- 172.21.15.78:0/1889357649 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f43700bd440 con 0x7f4378102230 2024-04-25T01:42:38.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.808+0000 7f437de6d640 1 -- 172.21.15.78:0/1889357649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f437802d170 con 0x7f4378102230 2024-04-25T01:42:38.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.809+0000 7f43757fa640 1 -- 172.21.15.78:0/1889357649 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f43700bcd40 con 0x7f4378102230 2024-04-25T01:42:38.798 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:42:38.798 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T01:42:38.798 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 1 2024-04-25T01:42:38.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.811+0000 7f437de6d640 1 -- 172.21.15.78:0/1889357649 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f43640a2c10 msgr2=0x7f43640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:38.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.811+0000 7f437de6d640 1 --2- 172.21.15.78:0/1889357649 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f43640a2c10 0x7f43640a50d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4368005fd0 tx=0x7f4368005850 comp rx=0 tx=0).stop 2024-04-25T01:42:38.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.811+0000 7f437de6d640 1 -- 172.21.15.78:0/1889357649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4378102230 msgr2=0x7f437819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:38.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.811+0000 7f437de6d640 1 --2- 172.21.15.78:0/1889357649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4378102230 0x7f437819ed00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4370005e30 tx=0x7f4370031ee0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.811+0000 7f437de6d640 1 -- 172.21.15.78:0/1889357649 shutdown_connections 2024-04-25T01:42:38.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.811+0000 7f437de6d640 1 --2- 172.21.15.78:0/1889357649 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f43640a2c10 0x7f43640a50d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.811+0000 7f437de6d640 1 --2- 172.21.15.78:0/1889357649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4378102230 0x7f437819ed00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.811+0000 7f437de6d640 1 --2- 172.21.15.78:0/1889357649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4378101870 0x7f437819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.811+0000 7f437de6d640 1 --2- 172.21.15.78:0/1889357649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4378100680 0x7f43780694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:38.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.811+0000 7f437de6d640 1 -- 172.21.15.78:0/1889357649 >> 172.21.15.78:0/1889357649 conn(0x7f43780fbbf0 msgr2=0x7f4378104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:38.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.811+0000 7f437de6d640 1 -- 172.21.15.78:0/1889357649 shutdown_connections 2024-04-25T01:42:38.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:38.811+0000 7f437de6d640 1 -- 172.21.15.78:0/1889357649 wait complete. 2024-04-25T01:42:39.114 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:38 smithi196 ceph-mon[32978]: Deploying daemon mds.l on smithi196 2024-04-25T01:42:39.114 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:38 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2928373156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:42:39.114 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:38 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2386937227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:42:39.114 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:38 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1889357649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:39.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:38 smithi078 ceph-mon[29469]: Deploying daemon mds.l on smithi196 2024-04-25T01:42:39.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:38 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2928373156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:42:39.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:38 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2386937227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:42:39.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:38 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1889357649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:39.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:38 smithi152 ceph-mon[32465]: Deploying daemon mds.l on smithi196 2024-04-25T01:42:39.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:38 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2928373156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:42:39.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:38 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2386937227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:42:39.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:38 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1889357649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:39.800 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:42:39 smithi196 systemd[1]: Starting Ceph mds.l for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:42:40.090 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:39 smithi196 ceph-mon[32978]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:40.090 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:42:39 smithi196 podman[63338]: 2024-04-25 01:42:39.714853869 +0000 UTC m=+0.018125224 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:42:40.090 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:42:39 smithi196 podman[63338]: 2024-04-25 01:42:39.866061693 +0000 UTC m=+0.169333042 container create e124f5eb5ddfe61e28a42a8e83f9b639314ccdd48efb9e8c68ef7c6b70aa7c9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T01:42:40.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:39 smithi078 ceph-mon[29469]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:40.295 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:39 smithi152 ceph-mon[32465]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:40.364 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:42:40 smithi196 podman[63338]: 2024-04-25 01:42:40.102096437 +0000 UTC m=+0.405367787 container init e124f5eb5ddfe61e28a42a8e83f9b639314ccdd48efb9e8c68ef7c6b70aa7c9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T01:42:40.364 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:42:40 smithi196 podman[63338]: 2024-04-25 01:42:40.104705091 +0000 UTC m=+0.407976444 container start e124f5eb5ddfe61e28a42a8e83f9b639314ccdd48efb9e8c68ef7c6b70aa7c9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:42:40.364 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:42:40 smithi196 ceph-mds[63356]: -- 172.21.15.196:0/697630723 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561cc6c4ba40 con 0x561cc6c1e800 2024-04-25T01:42:40.364 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:42:40 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-l[63352]: starting mds.l at 2024-04-25T01:42:40.364 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:42:40 smithi196 bash[63338]: e124f5eb5ddfe61e28a42a8e83f9b639314ccdd48efb9e8c68ef7c6b70aa7c9d 2024-04-25T01:42:40.364 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:42:40 smithi196 systemd[1]: Started Ceph mds.l for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:42:40.812 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:42:40.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.937+0000 7fc826f7e640 1 -- 172.21.15.78:0/365845703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820101870 msgr2=0x7fc820101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:40.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.937+0000 7fc826f7e640 1 --2- 172.21.15.78:0/365845703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820101870 0x7fc820101cf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc808009900 tx=0x7fc80802a410 comp rx=0 tx=0).stop 2024-04-25T01:42:40.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.937+0000 7fc826f7e640 1 -- 172.21.15.78:0/365845703 shutdown_connections 2024-04-25T01:42:40.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.937+0000 7fc826f7e640 1 --2- 172.21.15.78:0/365845703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc820102230 0x7fc820108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:40.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.937+0000 7fc826f7e640 1 --2- 172.21.15.78:0/365845703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820101870 0x7fc820101cf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:40.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.937+0000 7fc826f7e640 1 --2- 172.21.15.78:0/365845703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc820100680 0x7fc820100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:40.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.937+0000 7fc826f7e640 1 -- 172.21.15.78:0/365845703 >> 172.21.15.78:0/365845703 conn(0x7fc8200fbbf0 msgr2=0x7fc8200fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:40.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.937+0000 7fc826f7e640 1 -- 172.21.15.78:0/365845703 shutdown_connections 2024-04-25T01:42:40.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.937+0000 7fc826f7e640 1 -- 172.21.15.78:0/365845703 wait complete. 2024-04-25T01:42:40.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc826f7e640 1 Processor -- start 2024-04-25T01:42:40.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc826f7e640 1 -- start start 2024-04-25T01:42:40.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc826f7e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820100680 0x7fc820069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:40.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc826f7e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc820101870 0x7fc82019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:40.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc826f7e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc820102230 0x7fc82019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:40.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc826f7e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc82010c300 con 0x7fc820100680 2024-04-25T01:42:40.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc826f7e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc82010c180 con 0x7fc820102230 2024-04-25T01:42:40.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc826f7e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc82010c480 con 0x7fc820101870 2024-04-25T01:42:40.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc824d1a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820100680 0x7fc820069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:40.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc824d1a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820100680 0x7fc820069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55552/0 (socket says 172.21.15.78:55552) 2024-04-25T01:42:40.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc824d1a640 1 -- 172.21.15.78:0/1573955647 learned_addr learned my addr 172.21.15.78:0/1573955647 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:40.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc81ffff640 1 --2- 172.21.15.78:0/1573955647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc820101870 0x7fc82019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:40.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc82551b640 1 --2- 172.21.15.78:0/1573955647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc820102230 0x7fc82019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:40.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc824d1a640 1 -- 172.21.15.78:0/1573955647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc820101870 msgr2=0x7fc82019c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:40.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc824d1a640 1 --2- 172.21.15.78:0/1573955647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc820101870 0x7fc82019c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:40.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc824d1a640 1 -- 172.21.15.78:0/1573955647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc820102230 msgr2=0x7fc82019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:40.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc824d1a640 1 --2- 172.21.15.78:0/1573955647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc820102230 0x7fc82019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:40.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.938+0000 7fc824d1a640 1 -- 172.21.15.78:0/1573955647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82019f750 con 0x7fc820100680 2024-04-25T01:42:40.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.939+0000 7fc824d1a640 1 --2- 172.21.15.78:0/1573955647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820100680 0x7fc820069440 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc81000e9c0 tx=0x7fc81000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:40.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.939+0000 7fc81dffb640 1 -- 172.21.15.78:0/1573955647 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81000cd80 con 0x7fc820100680 2024-04-25T01:42:40.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.939+0000 7fc826f7e640 1 -- 172.21.15.78:0/1573955647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82019f9e0 con 0x7fc820100680 2024-04-25T01:42:40.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.939+0000 7fc81dffb640 1 -- 172.21.15.78:0/1573955647 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8100035f0 con 0x7fc820100680 2024-04-25T01:42:40.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.939+0000 7fc81dffb640 1 -- 172.21.15.78:0/1573955647 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc810010690 con 0x7fc820100680 2024-04-25T01:42:40.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.939+0000 7fc826f7e640 1 -- 172.21.15.78:0/1573955647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82019feb0 con 0x7fc820100680 2024-04-25T01:42:40.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.940+0000 7fc826f7e640 1 -- 172.21.15.78:0/1573955647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc820107a20 con 0x7fc820100680 2024-04-25T01:42:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.941+0000 7fc81dffb640 1 -- 172.21.15.78:0/1573955647 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc8100037a0 con 0x7fc820100680 2024-04-25T01:42:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.942+0000 7fc81dffb640 1 --2- 172.21.15.78:0/1573955647 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc80c0a2b90 0x7fc80c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.942+0000 7fc81ffff640 1 --2- 172.21.15.78:0/1573955647 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc80c0a2b90 0x7fc80c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.942+0000 7fc81dffb640 1 -- 172.21.15.78:0/1573955647 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc8100cefa0 con 0x7fc820100680 2024-04-25T01:42:40.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.942+0000 7fc81ffff640 1 --2- 172.21.15.78:0/1573955647 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc80c0a2b90 0x7fc80c0a5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc8080314e0 tx=0x7fc808000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:40.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:40.946+0000 7fc81dffb640 1 -- 172.21.15.78:0/1573955647 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc81009c3a0 con 0x7fc820100680 2024-04-25T01:42:41.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:41.163+0000 7fc826f7e640 1 -- 172.21.15.78:0/1573955647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc82002d170 con 0x7fc820100680 2024-04-25T01:42:41.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:41.163+0000 7fc81dffb640 1 -- 172.21.15.78:0/1573955647 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fc82002d170 con 0x7fc820100680 2024-04-25T01:42:41.153 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:42:41.153 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T01:42:41.153 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 2 2024-04-25T01:42:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:41.165+0000 7fc826f7e640 1 -- 172.21.15.78:0/1573955647 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc80c0a2b90 msgr2=0x7fc80c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:41.165+0000 7fc826f7e640 1 --2- 172.21.15.78:0/1573955647 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc80c0a2b90 0x7fc80c0a5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc8080314e0 tx=0x7fc808000bc0 comp rx=0 tx=0).stop 2024-04-25T01:42:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:41.165+0000 7fc826f7e640 1 -- 172.21.15.78:0/1573955647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820100680 msgr2=0x7fc820069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:41.165+0000 7fc826f7e640 1 --2- 172.21.15.78:0/1573955647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820100680 0x7fc820069440 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc81000e9c0 tx=0x7fc81000ee80 comp rx=0 tx=0).stop 2024-04-25T01:42:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:41.166+0000 7fc826f7e640 1 -- 172.21.15.78:0/1573955647 shutdown_connections 2024-04-25T01:42:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:41.166+0000 7fc826f7e640 1 --2- 172.21.15.78:0/1573955647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc820102230 0x7fc82019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:41.166+0000 7fc826f7e640 1 --2- 172.21.15.78:0/1573955647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc820101870 0x7fc82019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:41.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:41.166+0000 7fc826f7e640 1 --2- 172.21.15.78:0/1573955647 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc80c0a2b90 0x7fc80c0a5050 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:41.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:41.166+0000 7fc826f7e640 1 --2- 172.21.15.78:0/1573955647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc820100680 0x7fc820069440 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:41.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:41.166+0000 7fc826f7e640 1 -- 172.21.15.78:0/1573955647 >> 172.21.15.78:0/1573955647 conn(0x7fc8200fbbf0 msgr2=0x7fc820104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:41.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:41.166+0000 7fc826f7e640 1 -- 172.21.15.78:0/1573955647 shutdown_connections 2024-04-25T01:42:41.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:41.166+0000 7fc826f7e640 1 -- 172.21.15.78:0/1573955647 wait complete. 2024-04-25T01:42:41.195 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:41 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] up:boot 2024-04-25T01:42:41.196 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:41 smithi152 ceph-mon[32465]: fsmap 1 up:standby 2024-04-25T01:42:41.196 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:41 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T01:42:41.196 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:41 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:41.196 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:41 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:41.196 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:41 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:41.196 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:41 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:41.196 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:41 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:41.196 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:41 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:41.196 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:41 smithi152 ceph-mon[32465]: Deploying daemon mds.k on smithi152 2024-04-25T01:42:41.196 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:41 smithi152 ceph-mon[32465]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:41.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:41 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] up:boot 2024-04-25T01:42:41.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:41 smithi078 ceph-mon[29469]: fsmap 1 up:standby 2024-04-25T01:42:41.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T01:42:41.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:41.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:41.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:41.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:41.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:41.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:41 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:41.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:41 smithi078 ceph-mon[29469]: Deploying daemon mds.k on smithi152 2024-04-25T01:42:41.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:41 smithi078 ceph-mon[29469]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:41.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:41 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] up:boot 2024-04-25T01:42:41.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:41 smithi196 ceph-mon[32978]: fsmap 1 up:standby 2024-04-25T01:42:41.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:41 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T01:42:41.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:41 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:41.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:41 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:41.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:41 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:41.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:41 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:41.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:41 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:41.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:41 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:41.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:41 smithi196 ceph-mon[32978]: Deploying daemon mds.k on smithi152 2024-04-25T01:42:41.615 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:41 smithi196 ceph-mon[32978]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:42.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:42 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1573955647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:42.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:42 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1573955647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:42.613 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:42 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1573955647' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:42.698 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:42:42 smithi152 systemd[1]: Starting Ceph mds.k for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:42:42.961 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:42:42 smithi152 podman[61845]: 2024-04-25 01:42:42.612695399 +0000 UTC m=+0.018523590 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:42:42.961 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:42:42 smithi152 podman[61845]: 2024-04-25 01:42:42.741804454 +0000 UTC m=+0.147632632 container create 7d32f90d704370927879f6d7048b12a514624219a3bb49ca8ce11942fcf1970a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:42:42.962 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:42:42 smithi152 podman[61845]: 2024-04-25 01:42:42.973326159 +0000 UTC m=+0.379154344 container init 7d32f90d704370927879f6d7048b12a514624219a3bb49ca8ce11942fcf1970a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-k, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T01:42:43.167 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:42:43.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.289+0000 7f1d3f838640 1 -- 172.21.15.78:0/633785178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d38104b30 msgr2=0x7f1d38106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:43.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.289+0000 7f1d3f838640 1 --2- 172.21.15.78:0/633785178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d38104b30 0x7f1d38106f20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009900 tx=0x7f1d2802a410 comp rx=0 tx=0).stop 2024-04-25T01:42:43.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.289+0000 7f1d3f838640 1 -- 172.21.15.78:0/633785178 shutdown_connections 2024-04-25T01:42:43.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.289+0000 7f1d3f838640 1 --2- 172.21.15.78:0/633785178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d38107460 0x7f1d381098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:43.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.289+0000 7f1d3f838640 1 --2- 172.21.15.78:0/633785178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d38104b30 0x7f1d38106f20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:43.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.289+0000 7f1d3f838640 1 --2- 172.21.15.78:0/633785178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d38102200 0x7f1d381045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:43.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.290+0000 7f1d3f838640 1 -- 172.21.15.78:0/633785178 >> 172.21.15.78:0/633785178 conn(0x7f1d380fbbf0 msgr2=0x7f1d380fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:43.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.290+0000 7f1d3f838640 1 -- 172.21.15.78:0/633785178 shutdown_connections 2024-04-25T01:42:43.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.290+0000 7f1d3f838640 1 -- 172.21.15.78:0/633785178 wait complete. 2024-04-25T01:42:43.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.290+0000 7f1d3f838640 1 Processor -- start 2024-04-25T01:42:43.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.290+0000 7f1d3f838640 1 -- start start 2024-04-25T01:42:43.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3f838640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d38102200 0x7f1d38197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:43.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3f838640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d38104b30 0x7f1d38197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:43.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3f838640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d38107460 0x7f1d3819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:43.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3f838640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3810d3f0 con 0x7f1d38107460 2024-04-25T01:42:43.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3f838640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3810d270 con 0x7f1d38102200 2024-04-25T01:42:43.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3f838640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3810d570 con 0x7f1d38104b30 2024-04-25T01:42:43.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3ddd5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d38107460 0x7f1d3819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:43.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3ddd5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d38107460 0x7f1d3819f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55582/0 (socket says 172.21.15.78:55582) 2024-04-25T01:42:43.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3ddd5640 1 -- 172.21.15.78:0/1853434498 learned_addr learned my addr 172.21.15.78:0/1853434498 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:43.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3d5d4640 1 --2- 172.21.15.78:0/1853434498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d38102200 0x7f1d38197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:43.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3cdd3640 1 --2- 172.21.15.78:0/1853434498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d38104b30 0x7f1d38197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:43.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3ddd5640 1 -- 172.21.15.78:0/1853434498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d38104b30 msgr2=0x7f1d38197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:43.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3ddd5640 1 --2- 172.21.15.78:0/1853434498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d38104b30 0x7f1d38197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:43.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3ddd5640 1 -- 172.21.15.78:0/1853434498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d38102200 msgr2=0x7f1d38197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:43.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3ddd5640 1 --2- 172.21.15.78:0/1853434498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d38102200 0x7f1d38197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:43.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3ddd5640 1 -- 172.21.15.78:0/1853434498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3819f670 con 0x7f1d38107460 2024-04-25T01:42:43.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3ddd5640 1 --2- 172.21.15.78:0/1853434498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d38107460 0x7f1d3819f000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000cce0 tx=0x7f1d3000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:43.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d2e7fc640 1 -- 172.21.15.78:0/1853434498 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3000f450 con 0x7f1d38107460 2024-04-25T01:42:43.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d3f838640 1 -- 172.21.15.78:0/1853434498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3819f960 con 0x7f1d38107460 2024-04-25T01:42:43.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d2e7fc640 1 -- 172.21.15.78:0/1853434498 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d3000fad0 con 0x7f1d38107460 2024-04-25T01:42:43.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.291+0000 7f1d2e7fc640 1 -- 172.21.15.78:0/1853434498 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30018c10 con 0x7f1d38107460 2024-04-25T01:42:43.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.292+0000 7f1d3f838640 1 -- 172.21.15.78:0/1853434498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d3819fe60 con 0x7f1d38107460 2024-04-25T01:42:43.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.293+0000 7f1d3f838640 1 -- 172.21.15.78:0/1853434498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d380fd6e0 con 0x7f1d38107460 2024-04-25T01:42:43.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.294+0000 7f1d2e7fc640 1 -- 172.21.15.78:0/1853434498 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1d3000f5f0 con 0x7f1d38107460 2024-04-25T01:42:43.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.294+0000 7f1d2e7fc640 1 --2- 172.21.15.78:0/1853434498 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1d240a2b90 0x7f1d240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:43.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.294+0000 7f1d3d5d4640 1 --2- 172.21.15.78:0/1853434498 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1d240a2b90 0x7f1d240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:43.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.294+0000 7f1d2e7fc640 1 -- 172.21.15.78:0/1853434498 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1d300cf330 con 0x7f1d38107460 2024-04-25T01:42:43.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.295+0000 7f1d3d5d4640 1 --2- 172.21.15.78:0/1853434498 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1d240a2b90 0x7f1d240a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009b60 tx=0x7f1d34005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:43.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.299+0000 7f1d2e7fc640 1 -- 172.21.15.78:0/1853434498 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1d30014030 con 0x7f1d38107460 2024-04-25T01:42:43.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:43 smithi152 ceph-mon[32465]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:43.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:43 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:43.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:43 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:43.296 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:42:42 smithi152 podman[61845]: 2024-04-25 01:42:42.975999866 +0000 UTC m=+0.381828045 container start 7d32f90d704370927879f6d7048b12a514624219a3bb49ca8ce11942fcf1970a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-k, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T01:42:43.296 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:42:42 smithi152 ceph-mds[61864]: -- 172.21.15.152:0/207413535 <== mon.2 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624eef83a40 con 0x5624eef57000 2024-04-25T01:42:43.296 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:42:43 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-k[61860]: starting mds.k at 2024-04-25T01:42:43.296 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:42:43 smithi152 bash[61845]: 7d32f90d704370927879f6d7048b12a514624219a3bb49ca8ce11942fcf1970a 2024-04-25T01:42:43.296 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:42:43 smithi152 systemd[1]: Started Ceph mds.k for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:42:43.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:43 smithi078 ceph-mon[29469]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:43.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:43 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:43.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:43 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:43.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.540+0000 7f1d3f838640 1 -- 172.21.15.78:0/1853434498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d3802d320 con 0x7f1d38107460 2024-04-25T01:42:43.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.541+0000 7f1d2e7fc640 1 -- 172.21.15.78:0/1853434498 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f1d3009c730 con 0x7f1d38107460 2024-04-25T01:42:43.530 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:42:43.530 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T01:42:43.530 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 3 2024-04-25T01:42:43.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.543+0000 7f1d3f838640 1 -- 172.21.15.78:0/1853434498 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1d240a2b90 msgr2=0x7f1d240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:43.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.543+0000 7f1d3f838640 1 --2- 172.21.15.78:0/1853434498 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1d240a2b90 0x7f1d240a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009b60 tx=0x7f1d34005f90 comp rx=0 tx=0).stop 2024-04-25T01:42:43.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.543+0000 7f1d3f838640 1 -- 172.21.15.78:0/1853434498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d38107460 msgr2=0x7f1d3819f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.543+0000 7f1d3f838640 1 --2- 172.21.15.78:0/1853434498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d38107460 0x7f1d3819f000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000cce0 tx=0x7f1d3000a620 comp rx=0 tx=0).stop 2024-04-25T01:42:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.543+0000 7f1d3f838640 1 -- 172.21.15.78:0/1853434498 shutdown_connections 2024-04-25T01:42:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.543+0000 7f1d3f838640 1 --2- 172.21.15.78:0/1853434498 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1d240a2b90 0x7f1d240a5050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.543+0000 7f1d3f838640 1 --2- 172.21.15.78:0/1853434498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d38107460 0x7f1d3819f000 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.543+0000 7f1d3f838640 1 --2- 172.21.15.78:0/1853434498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d38104b30 0x7f1d38197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.543+0000 7f1d3f838640 1 --2- 172.21.15.78:0/1853434498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d38102200 0x7f1d38197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.543+0000 7f1d3f838640 1 -- 172.21.15.78:0/1853434498 >> 172.21.15.78:0/1853434498 conn(0x7f1d380fbbf0 msgr2=0x7f1d38102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.543+0000 7f1d3f838640 1 -- 172.21.15.78:0/1853434498 shutdown_connections 2024-04-25T01:42:43.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:43.543+0000 7f1d3f838640 1 -- 172.21.15.78:0/1853434498 wait complete. 2024-04-25T01:42:43.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:43 smithi196 ceph-mon[32978]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:43.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:43 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:43.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:43 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:44.381 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:44 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] up:boot 2024-04-25T01:42:44.381 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:44 smithi152 ceph-mon[32465]: fsmap 2 up:standby 2024-04-25T01:42:44.381 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:44 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T01:42:44.382 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:44 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:44.382 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:44 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:44.382 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:44 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:44.382 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:44 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:44.382 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:44 smithi152 ceph-mon[32465]: Deploying daemon mds.j on smithi078 2024-04-25T01:42:44.382 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:44 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1853434498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:44.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:44 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] up:boot 2024-04-25T01:42:44.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:44 smithi078 ceph-mon[29469]: fsmap 2 up:standby 2024-04-25T01:42:44.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:44 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T01:42:44.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:44 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:44.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:44 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:44.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:44 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:44.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:44 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:44.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:44 smithi078 ceph-mon[29469]: Deploying daemon mds.j on smithi078 2024-04-25T01:42:44.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:44 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1853434498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:44.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:44 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] up:boot 2024-04-25T01:42:44.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:44 smithi196 ceph-mon[32978]: fsmap 2 up:standby 2024-04-25T01:42:44.615 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:44 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T01:42:44.615 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:44 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:44.615 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:44 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:44.615 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:44 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:44.615 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:44 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:44.615 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:44 smithi196 ceph-mon[32978]: Deploying daemon mds.j on smithi078 2024-04-25T01:42:44.615 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:44 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1853434498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:45.405 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:45 smithi078 ceph-mon[29469]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:45.406 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:42:45 smithi078 systemd[1]: Starting Ceph mds.j for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:42:45.544 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:42:45.546 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:45 smithi152 ceph-mon[32465]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:45.613 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:45 smithi196 ceph-mon[32978]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.657+0000 7f36fad9e640 1 -- 172.21.15.78:0/62928141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f41016b0 msgr2=0x7f36f4101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.657+0000 7f36fad9e640 1 --2- 172.21.15.78:0/62928141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f41016b0 0x7f36f4101b30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009900 tx=0x7f36e402a410 comp rx=0 tx=0).stop 2024-04-25T01:42:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.657+0000 7f36fad9e640 1 -- 172.21.15.78:0/62928141 shutdown_connections 2024-04-25T01:42:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.657+0000 7f36fad9e640 1 --2- 172.21.15.78:0/62928141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f4102070 0x7f36f4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.657+0000 7f36fad9e640 1 --2- 172.21.15.78:0/62928141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f41016b0 0x7f36f4101b30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.657+0000 7f36fad9e640 1 --2- 172.21.15.78:0/62928141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36f41004c0 0x7f36f41008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.657+0000 7f36fad9e640 1 -- 172.21.15.78:0/62928141 >> 172.21.15.78:0/62928141 conn(0x7f36f40fba50 msgr2=0x7f36f40fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.657+0000 7f36fad9e640 1 -- 172.21.15.78:0/62928141 shutdown_connections 2024-04-25T01:42:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.657+0000 7f36fad9e640 1 -- 172.21.15.78:0/62928141 wait complete. 2024-04-25T01:42:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36fad9e640 1 Processor -- start 2024-04-25T01:42:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36fad9e640 1 -- start start 2024-04-25T01:42:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36fad9e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f41004c0 0x7f36f41978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36fad9e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f41016b0 0x7f36f4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36fad9e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36f4102070 0x7f36f419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36fad9e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36f410d260 con 0x7f36f41004c0 2024-04-25T01:42:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36fad9e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f36f410d0e0 con 0x7f36f4102070 2024-04-25T01:42:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36fad9e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36f410d3e0 con 0x7f36f41016b0 2024-04-25T01:42:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36f9d9c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f41004c0 0x7f36f41978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36f9d9c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f41004c0 0x7f36f41978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55602/0 (socket says 172.21.15.78:55602) 2024-04-25T01:42:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36f9d9c640 1 -- 172.21.15.78:0/1598219265 learned_addr learned my addr 172.21.15.78:0/1598219265 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36fa59d640 1 --2- 172.21.15.78:0/1598219265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36f4102070 0x7f36f419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36f959b640 1 --2- 172.21.15.78:0/1598219265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f41016b0 0x7f36f4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:45.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36f9d9c640 1 -- 172.21.15.78:0/1598219265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f41016b0 msgr2=0x7f36f4197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36f9d9c640 1 --2- 172.21.15.78:0/1598219265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f41016b0 0x7f36f4197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36f9d9c640 1 -- 172.21.15.78:0/1598219265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36f4102070 msgr2=0x7f36f419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36f9d9c640 1 --2- 172.21.15.78:0/1598219265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36f4102070 0x7f36f419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.658+0000 7f36f9d9c640 1 -- 172.21.15.78:0/1598219265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f419f500 con 0x7f36f41004c0 2024-04-25T01:42:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.659+0000 7f36f9d9c640 1 --2- 172.21.15.78:0/1598219265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f41004c0 0x7f36f41978e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f36f000e9c0 tx=0x7f36f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.659+0000 7f36eaffd640 1 -- 172.21.15.78:0/1598219265 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f000cd80 con 0x7f36f41004c0 2024-04-25T01:42:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.659+0000 7f36eaffd640 1 -- 172.21.15.78:0/1598219265 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f00035f0 con 0x7f36f41004c0 2024-04-25T01:42:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.659+0000 7f36fad9e640 1 -- 172.21.15.78:0/1598219265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f419f7f0 con 0x7f36f41004c0 2024-04-25T01:42:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.659+0000 7f36eaffd640 1 -- 172.21.15.78:0/1598219265 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0010690 con 0x7f36f41004c0 2024-04-25T01:42:45.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.659+0000 7f36fad9e640 1 -- 172.21.15.78:0/1598219265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f419fd40 con 0x7f36f41004c0 2024-04-25T01:42:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.660+0000 7f36fad9e640 1 -- 172.21.15.78:0/1598219265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f4107830 con 0x7f36f41004c0 2024-04-25T01:42:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.661+0000 7f36eaffd640 1 -- 172.21.15.78:0/1598219265 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f36f00037a0 con 0x7f36f41004c0 2024-04-25T01:42:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.662+0000 7f36eaffd640 1 --2- 172.21.15.78:0/1598219265 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36e00a2b90 0x7f36e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:45.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.662+0000 7f36f959b640 1 --2- 172.21.15.78:0/1598219265 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36e00a2b90 0x7f36e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:45.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.662+0000 7f36eaffd640 1 -- 172.21.15.78:0/1598219265 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f36f00cf0d0 con 0x7f36f41004c0 2024-04-25T01:42:45.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.662+0000 7f36f959b640 1 --2- 172.21.15.78:0/1598219265 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36e00a2b90 0x7f36e00a5050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f36e4031610 tx=0x7f36e4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:45.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.667+0000 7f36eaffd640 1 -- 172.21.15.78:0/1598219265 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f36f009c430 con 0x7f36f41004c0 2024-04-25T01:42:45.754 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:42:45 smithi078 podman[67100]: 2024-04-25 01:42:45.538731267 +0000 UTC m=+0.018015730 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:42:45.754 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:42:45 smithi078 podman[67100]: 2024-04-25 01:42:45.663968884 +0000 UTC m=+0.143253335 container create 9a1216143646353ca9b984db4adea23861975affd340e8b309c488837962100f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:42:45.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.886+0000 7f36fad9e640 1 -- 172.21.15.78:0/1598219265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36f402d170 con 0x7f36f41004c0 2024-04-25T01:42:45.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.886+0000 7f36eaffd640 1 -- 172.21.15.78:0/1598219265 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f36f009bd30 con 0x7f36f41004c0 2024-04-25T01:42:45.876 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:42:45.876 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T01:42:45.876 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 3 2024-04-25T01:42:45.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.888+0000 7f36fad9e640 1 -- 172.21.15.78:0/1598219265 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36e00a2b90 msgr2=0x7f36e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:45.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.888+0000 7f36fad9e640 1 --2- 172.21.15.78:0/1598219265 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36e00a2b90 0x7f36e00a5050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f36e4031610 tx=0x7f36e4000bc0 comp rx=0 tx=0).stop 2024-04-25T01:42:45.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.888+0000 7f36fad9e640 1 -- 172.21.15.78:0/1598219265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f41004c0 msgr2=0x7f36f41978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:45.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.888+0000 7f36fad9e640 1 --2- 172.21.15.78:0/1598219265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f41004c0 0x7f36f41978e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f36f000e9c0 tx=0x7f36f000ee80 comp rx=0 tx=0).stop 2024-04-25T01:42:45.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.888+0000 7f36fad9e640 1 -- 172.21.15.78:0/1598219265 shutdown_connections 2024-04-25T01:42:45.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.888+0000 7f36fad9e640 1 --2- 172.21.15.78:0/1598219265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36f4102070 0x7f36f419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:45.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.889+0000 7f36fad9e640 1 --2- 172.21.15.78:0/1598219265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36f41016b0 0x7f36f4197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:45.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.889+0000 7f36fad9e640 1 --2- 172.21.15.78:0/1598219265 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36e00a2b90 0x7f36e00a5050 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:45.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.889+0000 7f36fad9e640 1 --2- 172.21.15.78:0/1598219265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36f41004c0 0x7f36f41978e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:45.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.889+0000 7f36fad9e640 1 -- 172.21.15.78:0/1598219265 >> 172.21.15.78:0/1598219265 conn(0x7f36f40fba50 msgr2=0x7f36f4104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:45.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.889+0000 7f36fad9e640 1 -- 172.21.15.78:0/1598219265 shutdown_connections 2024-04-25T01:42:45.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:45.889+0000 7f36fad9e640 1 -- 172.21.15.78:0/1598219265 wait complete. 2024-04-25T01:42:46.147 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:42:45 smithi078 podman[67100]: 2024-04-25 01:42:45.927774288 +0000 UTC m=+0.407058747 container init 9a1216143646353ca9b984db4adea23861975affd340e8b309c488837962100f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T01:42:46.147 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:42:45 smithi078 podman[67100]: 2024-04-25 01:42:45.930469409 +0000 UTC m=+0.409753869 container start 9a1216143646353ca9b984db4adea23861975affd340e8b309c488837962100f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-25T01:42:46.147 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:42:45 smithi078 ceph-mds[67160]: -- 172.21.15.78:0/3484383786 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555abeea7a40 con 0x555abee7ac00 2024-04-25T01:42:46.147 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:42:45 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-j[67154]: starting mds.j at 2024-04-25T01:42:46.147 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:42:46 smithi078 bash[67100]: 9a1216143646353ca9b984db4adea23861975affd340e8b309c488837962100f 2024-04-25T01:42:46.147 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:42:46 smithi078 systemd[1]: Started Ceph mds.j for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:42:46.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:46 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1598219265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:46.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:46 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:46.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:46 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:46.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:46 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:46.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:46 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:46.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:46 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:46.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:46 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:46.546 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:46 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1598219265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:46.546 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:46 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:46.546 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:46 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:46.546 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:46 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:46.546 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:46 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:46.546 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:46 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:46.547 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:46 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:46.613 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:46 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1598219265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:46.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:46 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:46.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:46 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:46.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:46 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:46.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:46 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:46.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:46 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:46.614 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:46 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:47.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:47 smithi078 ceph-mon[29469]: Deploying daemon mds.d on smithi078 2024-04-25T01:42:47.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:47 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.78:6834/2013480406,v1:172.21.15.78:6835/2013480406] up:boot 2024-04-25T01:42:47.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:47 smithi078 ceph-mon[29469]: fsmap 3 up:standby 2024-04-25T01:42:47.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:47 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T01:42:47.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:47 smithi078 ceph-mon[29469]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:47.522 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:47 smithi196 ceph-mon[32978]: Deploying daemon mds.d on smithi078 2024-04-25T01:42:47.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:47 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.78:6834/2013480406,v1:172.21.15.78:6835/2013480406] up:boot 2024-04-25T01:42:47.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:47 smithi196 ceph-mon[32978]: fsmap 3 up:standby 2024-04-25T01:42:47.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:47 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T01:42:47.523 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:47 smithi196 ceph-mon[32978]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:47.546 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:47 smithi152 ceph-mon[32465]: Deploying daemon mds.d on smithi078 2024-04-25T01:42:47.546 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:47 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.78:6834/2013480406,v1:172.21.15.78:6835/2013480406] up:boot 2024-04-25T01:42:47.546 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:47 smithi152 ceph-mon[32465]: fsmap 3 up:standby 2024-04-25T01:42:47.546 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:47 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T01:42:47.546 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:47 smithi152 ceph-mon[32465]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:47.890 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:42:48.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.028+0000 7f172eb09640 1 -- 172.21.15.78:0/1123942366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1728102230 msgr2=0x7f1728108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:48.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.028+0000 7f172eb09640 1 --2- 172.21.15.78:0/1123942366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1728102230 0x7f1728108ac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1710009900 tx=0x7f171002a410 comp rx=0 tx=0).stop 2024-04-25T01:42:48.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.028+0000 7f172eb09640 1 -- 172.21.15.78:0/1123942366 shutdown_connections 2024-04-25T01:42:48.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.028+0000 7f172eb09640 1 --2- 172.21.15.78:0/1123942366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1728102230 0x7f1728108ac0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:48.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.028+0000 7f172eb09640 1 --2- 172.21.15.78:0/1123942366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1728101870 0x7f1728101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:48.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.028+0000 7f172eb09640 1 --2- 172.21.15.78:0/1123942366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1728100680 0x7f1728100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:48.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.028+0000 7f172eb09640 1 -- 172.21.15.78:0/1123942366 >> 172.21.15.78:0/1123942366 conn(0x7f17280fbbf0 msgr2=0x7f17280fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:48.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.028+0000 7f172eb09640 1 -- 172.21.15.78:0/1123942366 shutdown_connections 2024-04-25T01:42:48.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.028+0000 7f172eb09640 1 -- 172.21.15.78:0/1123942366 wait complete. 2024-04-25T01:42:48.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.029+0000 7f172eb09640 1 Processor -- start 2024-04-25T01:42:48.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f172eb09640 1 -- start start 2024-04-25T01:42:48.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f172eb09640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1728100680 0x7f1728069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:48.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f172eb09640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1728101870 0x7f172819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:48.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f172eb09640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1728102230 0x7f172819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:48.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f172eb09640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f172810c3f0 con 0x7f1728101870 2024-04-25T01:42:48.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f172eb09640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f172810c270 con 0x7f1728100680 2024-04-25T01:42:48.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f172eb09640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f172810c570 con 0x7f1728102230 2024-04-25T01:42:48.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f1727fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1728101870 0x7f172819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:48.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f1727fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1728101870 0x7f172819c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55632/0 (socket says 172.21.15.78:55632) 2024-04-25T01:42:48.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f1727fff640 1 -- 172.21.15.78:0/597901142 learned_addr learned my addr 172.21.15.78:0/597901142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:48.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f172d0a6640 1 --2- 172.21.15.78:0/597901142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1728102230 0x7f172819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:48.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f1727fff640 1 -- 172.21.15.78:0/597901142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1728102230 msgr2=0x7f172819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:48.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f1727fff640 1 --2- 172.21.15.78:0/597901142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1728102230 0x7f172819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:48.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f1727fff640 1 -- 172.21.15.78:0/597901142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1728100680 msgr2=0x7f1728069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:48.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f1727fff640 1 --2- 172.21.15.78:0/597901142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1728100680 0x7f1728069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:48.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f1727fff640 1 -- 172.21.15.78:0/597901142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172819f7d0 con 0x7f1728101870 2024-04-25T01:42:48.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f1727fff640 1 --2- 172.21.15.78:0/597901142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1728101870 0x7f172819c8d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f171800ba50 tx=0x7f171800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:48.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f1725ffb640 1 -- 172.21.15.78:0/597901142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718019070 con 0x7f1728101870 2024-04-25T01:42:48.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f1725ffb640 1 -- 172.21.15.78:0/597901142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f171800cbe0 con 0x7f1728101870 2024-04-25T01:42:48.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f172eb09640 1 -- 172.21.15.78:0/597901142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172819fac0 con 0x7f1728101870 2024-04-25T01:42:48.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.030+0000 7f1725ffb640 1 -- 172.21.15.78:0/597901142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17180076e0 con 0x7f1728101870 2024-04-25T01:42:48.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.031+0000 7f172eb09640 1 -- 172.21.15.78:0/597901142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17281a0010 con 0x7f1728101870 2024-04-25T01:42:48.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.032+0000 7f172eb09640 1 -- 172.21.15.78:0/597901142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17281079a0 con 0x7f1728101870 2024-04-25T01:42:48.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.032+0000 7f1725ffb640 1 -- 172.21.15.78:0/597901142 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f171801c070 con 0x7f1728101870 2024-04-25T01:42:48.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.033+0000 7f1725ffb640 1 --2- 172.21.15.78:0/597901142 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17140a2b90 0x7f17140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:48.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.033+0000 7f172c8a5640 1 --2- 172.21.15.78:0/597901142 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17140a2b90 0x7f17140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:48.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.033+0000 7f1725ffb640 1 -- 172.21.15.78:0/597901142 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f17180ce2a0 con 0x7f1728101870 2024-04-25T01:42:48.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.034+0000 7f172c8a5640 1 --2- 172.21.15.78:0/597901142 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17140a2b90 0x7f17140a5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f17281016d0 tx=0x7f1720006c90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:48.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.038+0000 7f1725ffb640 1 -- 172.21.15.78:0/597901142 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f171809b600 con 0x7f1728101870 2024-04-25T01:42:48.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.266+0000 7f172eb09640 1 -- 172.21.15.78:0/597901142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1728062340 con 0x7f1728101870 2024-04-25T01:42:48.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.266+0000 7f1725ffb640 1 -- 172.21.15.78:0/597901142 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f171809af00 con 0x7f1728101870 2024-04-25T01:42:48.256 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:42:48.256 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T01:42:48.256 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 4 2024-04-25T01:42:48.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.268+0000 7f172eb09640 1 -- 172.21.15.78:0/597901142 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17140a2b90 msgr2=0x7f17140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:48.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.268+0000 7f172eb09640 1 --2- 172.21.15.78:0/597901142 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17140a2b90 0x7f17140a5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f17281016d0 tx=0x7f1720006c90 comp rx=0 tx=0).stop 2024-04-25T01:42:48.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.268+0000 7f172eb09640 1 -- 172.21.15.78:0/597901142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1728101870 msgr2=0x7f172819c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:48.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.268+0000 7f172eb09640 1 --2- 172.21.15.78:0/597901142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1728101870 0x7f172819c8d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f171800ba50 tx=0x7f171800bf10 comp rx=0 tx=0).stop 2024-04-25T01:42:48.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.268+0000 7f172eb09640 1 -- 172.21.15.78:0/597901142 shutdown_connections 2024-04-25T01:42:48.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.268+0000 7f172eb09640 1 --2- 172.21.15.78:0/597901142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1728102230 0x7f172819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:48.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.268+0000 7f172eb09640 1 --2- 172.21.15.78:0/597901142 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17140a2b90 0x7f17140a5050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:48.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.268+0000 7f172eb09640 1 --2- 172.21.15.78:0/597901142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1728101870 0x7f172819c8d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:48.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.268+0000 7f172eb09640 1 --2- 172.21.15.78:0/597901142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1728100680 0x7f1728069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:48.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.268+0000 7f172eb09640 1 -- 172.21.15.78:0/597901142 >> 172.21.15.78:0/597901142 conn(0x7f17280fbbf0 msgr2=0x7f1728104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:48.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.269+0000 7f172eb09640 1 -- 172.21.15.78:0/597901142 shutdown_connections 2024-04-25T01:42:48.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:48.269+0000 7f172eb09640 1 -- 172.21.15.78:0/597901142 wait complete. 2024-04-25T01:42:48.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:48 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/597901142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:48.710 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:42:48 smithi078 systemd[1]: Starting Ceph mds.d for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:42:48.711 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:48 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/597901142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:48.787 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:42:48 smithi078 podman[67492]: 2024-04-25 01:42:48.449407152 +0000 UTC m=+0.017210113 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:42:48.787 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:42:48 smithi078 podman[67492]: 2024-04-25 01:42:48.553369891 +0000 UTC m=+0.121172881 container create 81055bfaff020ed170910cafc700c8dd59a9d1f044b0f803f4a8f88d4debff44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-d, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-25T01:42:48.787 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:42:48 smithi078 podman[67492]: 2024-04-25 01:42:48.797922648 +0000 UTC m=+0.365725594 container init 81055bfaff020ed170910cafc700c8dd59a9d1f044b0f803f4a8f88d4debff44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T01:42:48.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:48 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/597901142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:49.254 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:42:48 smithi078 podman[67492]: 2024-04-25 01:42:48.800800991 +0000 UTC m=+0.368603937 container start 81055bfaff020ed170910cafc700c8dd59a9d1f044b0f803f4a8f88d4debff44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:42:49.254 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:42:48 smithi078 ceph-mds[67511]: -- 172.21.15.78:0/805176103 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa9eaf7a40 con 0x55fa9eacb000 2024-04-25T01:42:49.255 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:42:48 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-d[67507]: starting mds.d at 2024-04-25T01:42:49.255 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:42:48 smithi078 bash[67492]: 81055bfaff020ed170910cafc700c8dd59a9d1f044b0f803f4a8f88d4debff44 2024-04-25T01:42:49.255 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:42:48 smithi078 systemd[1]: Started Ceph mds.d for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:42:50.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:49 smithi078 ceph-mon[29469]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:50.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:50.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:50.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:50.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:50.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:50.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:50.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:49 smithi078 ceph-mon[29469]: Deploying daemon mds.e on smithi152 2024-04-25T01:42:50.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:49 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.78:6836/3364816743,v1:172.21.15.78:6837/3364816743] up:boot 2024-04-25T01:42:50.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:49 smithi078 ceph-mon[29469]: fsmap 4 up:standby 2024-04-25T01:42:50.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T01:42:50.270 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:42:50.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:49 smithi152 ceph-mon[32465]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:50.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:50.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:50.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:50.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:50.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:50.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:50.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:49 smithi152 ceph-mon[32465]: Deploying daemon mds.e on smithi152 2024-04-25T01:42:50.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:49 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.78:6836/3364816743,v1:172.21.15.78:6837/3364816743] up:boot 2024-04-25T01:42:50.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:49 smithi152 ceph-mon[32465]: fsmap 4 up:standby 2024-04-25T01:42:50.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T01:42:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:49 smithi196 ceph-mon[32978]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:50.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:49 smithi196 ceph-mon[32978]: Deploying daemon mds.e on smithi152 2024-04-25T01:42:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:49 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.78:6836/3364816743,v1:172.21.15.78:6837/3364816743] up:boot 2024-04-25T01:42:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:49 smithi196 ceph-mon[32978]: fsmap 4 up:standby 2024-04-25T01:42:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T01:42:50.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.392+0000 7f297c688640 1 -- 172.21.15.78:0/405104026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2974101890 msgr2=0x7f2974101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:50.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.392+0000 7f297c688640 1 --2- 172.21.15.78:0/405104026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2974101890 0x7f2974101d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2964009900 tx=0x7f296402a410 comp rx=0 tx=0).stop 2024-04-25T01:42:50.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.392+0000 7f297c688640 1 -- 172.21.15.78:0/405104026 shutdown_connections 2024-04-25T01:42:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.392+0000 7f297c688640 1 --2- 172.21.15.78:0/405104026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2974102250 0x7f2974108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.392+0000 7f297c688640 1 --2- 172.21.15.78:0/405104026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2974101890 0x7f2974101d10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.392+0000 7f297c688640 1 --2- 172.21.15.78:0/405104026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29741006a0 0x7f2974100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.392+0000 7f297c688640 1 -- 172.21.15.78:0/405104026 >> 172.21.15.78:0/405104026 conn(0x7f29740fbc30 msgr2=0x7f29740fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.392+0000 7f297c688640 1 -- 172.21.15.78:0/405104026 shutdown_connections 2024-04-25T01:42:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.392+0000 7f297c688640 1 -- 172.21.15.78:0/405104026 wait complete. 2024-04-25T01:42:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297c688640 1 Processor -- start 2024-04-25T01:42:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297c688640 1 -- start start 2024-04-25T01:42:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297c688640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29741006a0 0x7f2974197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297c688640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2974101890 0x7f2974198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297c688640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2974102250 0x7f297419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297c688640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f297410d430 con 0x7f2974102250 2024-04-25T01:42:50.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297c688640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f297410d2b0 con 0x7f2974101890 2024-04-25T01:42:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297c688640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f297410d5b0 con 0x7f29741006a0 2024-04-25T01:42:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297ac25640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2974102250 0x7f297419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297ac25640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2974102250 0x7f297419f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39130/0 (socket says 172.21.15.78:39130) 2024-04-25T01:42:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297ac25640 1 -- 172.21.15.78:0/569868994 learned_addr learned my addr 172.21.15.78:0/569868994 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297a424640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29741006a0 0x7f2974197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f2979c23640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2974101890 0x7f2974198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297ac25640 1 -- 172.21.15.78:0/569868994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29741006a0 msgr2=0x7f2974197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297ac25640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29741006a0 0x7f2974197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297ac25640 1 -- 172.21.15.78:0/569868994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2974101890 msgr2=0x7f2974198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297ac25640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2974101890 0x7f2974198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297ac25640 1 -- 172.21.15.78:0/569868994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297419f6e0 con 0x7f2974102250 2024-04-25T01:42:50.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f2979c23640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2974101890 0x7f2974198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:42:50.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.393+0000 7f297ac25640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2974102250 0x7f297419f070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f296c009fd0 tx=0x7f296c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:50.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.394+0000 7f296b7fe640 1 -- 172.21.15.78:0/569868994 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c017070 con 0x7f2974102250 2024-04-25T01:42:50.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.394+0000 7f297c688640 1 -- 172.21.15.78:0/569868994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297419f9d0 con 0x7f2974102250 2024-04-25T01:42:50.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.394+0000 7f296b7fe640 1 -- 172.21.15.78:0/569868994 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f296c003e50 con 0x7f2974102250 2024-04-25T01:42:50.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.394+0000 7f296b7fe640 1 -- 172.21.15.78:0/569868994 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c0106c0 con 0x7f2974102250 2024-04-25T01:42:50.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.394+0000 7f297c688640 1 -- 172.21.15.78:0/569868994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f297419fed0 con 0x7f2974102250 2024-04-25T01:42:50.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.395+0000 7f297c688640 1 -- 172.21.15.78:0/569868994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29741079e0 con 0x7f2974102250 2024-04-25T01:42:50.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.396+0000 7f296b7fe640 1 -- 172.21.15.78:0/569868994 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f296c003970 con 0x7f2974102250 2024-04-25T01:42:50.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.396+0000 7f296b7fe640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29600a2b90 0x7f29600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:50.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.396+0000 7f297a424640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29600a2b90 0x7f29600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:50.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.396+0000 7f296b7fe640 1 -- 172.21.15.78:0/569868994 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f296c0ce960 con 0x7f2974102250 2024-04-25T01:42:50.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.397+0000 7f297a424640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29600a2b90 0x7f29600a5050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f29741016f0 tx=0x7f2970005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:50.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.401+0000 7f296b7fe640 1 -- 172.21.15.78:0/569868994 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f296c09bd60 con 0x7f2974102250 2024-04-25T01:42:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.611+0000 7f297c688640 1 -- 172.21.15.78:0/569868994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f297402d170 con 0x7f2974102250 2024-04-25T01:42:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.612+0000 7f296b7fe640 1 -- 172.21.15.78:0/569868994 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f296c09b660 con 0x7f2974102250 2024-04-25T01:42:50.602 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:42:50.602 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T01:42:50.602 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 5 2024-04-25T01:42:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.614+0000 7f297c688640 1 -- 172.21.15.78:0/569868994 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29600a2b90 msgr2=0x7f29600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:50.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.614+0000 7f297c688640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29600a2b90 0x7f29600a5050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f29741016f0 tx=0x7f2970005f90 comp rx=0 tx=0).stop 2024-04-25T01:42:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.614+0000 7f297c688640 1 -- 172.21.15.78:0/569868994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2974102250 msgr2=0x7f297419f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.614+0000 7f297c688640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2974102250 0x7f297419f070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f296c009fd0 tx=0x7f296c00b260 comp rx=0 tx=0).stop 2024-04-25T01:42:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.614+0000 7f297c688640 1 -- 172.21.15.78:0/569868994 shutdown_connections 2024-04-25T01:42:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.614+0000 7f297c688640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29600a2b90 0x7f29600a5050 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.614+0000 7f297c688640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2974102250 0x7f297419f070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.614+0000 7f297c688640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2974101890 0x7f2974198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.614+0000 7f297c688640 1 --2- 172.21.15.78:0/569868994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29741006a0 0x7f2974197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.614+0000 7f297c688640 1 -- 172.21.15.78:0/569868994 >> 172.21.15.78:0/569868994 conn(0x7f29740fbc30 msgr2=0x7f2974104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.614+0000 7f297c688640 1 -- 172.21.15.78:0/569868994 shutdown_connections 2024-04-25T01:42:50.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:50.614+0000 7f297c688640 1 -- 172.21.15.78:0/569868994 wait complete. 2024-04-25T01:42:51.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:50 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/569868994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:51.275 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:50 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/569868994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:51.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:50 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/569868994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:51.546 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:42:51 smithi152 podman[62163]: 2024-04-25 01:42:51.528378479 +0000 UTC m=+0.138381473 container create ebc5baaea7ef0aa1615997a1f13bcfb8c1a01e7b0ec0e1decdb18cd4b66cccde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T01:42:51.948 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:42:51 smithi152 podman[62163]: 2024-04-25 01:42:51.797984051 +0000 UTC m=+0.407987047 container init ebc5baaea7ef0aa1615997a1f13bcfb8c1a01e7b0ec0e1decdb18cd4b66cccde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T01:42:51.948 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:42:51 smithi152 podman[62163]: 2024-04-25 01:42:51.800546969 +0000 UTC m=+0.410549972 container start ebc5baaea7ef0aa1615997a1f13bcfb8c1a01e7b0ec0e1decdb18cd4b66cccde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:42:51.948 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:42:51 smithi152 ceph-mds[62182]: -- 172.21.15.152:0/3966694330 <== mon.2 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56222091da40 con 0x5622208f0800 2024-04-25T01:42:51.949 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:42:51 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-e[62178]: starting mds.e at 2024-04-25T01:42:51.949 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:42:51 smithi152 bash[62163]: ebc5baaea7ef0aa1615997a1f13bcfb8c1a01e7b0ec0e1decdb18cd4b66cccde 2024-04-25T01:42:51.949 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:42:51 smithi152 systemd[1]: Started Ceph mds.e for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:42:52.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:51 smithi078 ceph-mon[29469]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:52.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:52.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:51 smithi152 ceph-mon[32465]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:52.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:52.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:51 smithi196 ceph-mon[32978]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:52.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:52.615 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:42:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.738+0000 7fea2b75a640 1 -- 172.21.15.78:0/490703748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea24102980 msgr2=0x7fea24102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:52.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.738+0000 7fea2b75a640 1 --2- 172.21.15.78:0/490703748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea24102980 0x7fea24102e00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fea14009900 tx=0x7fea1402a410 comp rx=0 tx=0).stop 2024-04-25T01:42:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.738+0000 7fea2b75a640 1 -- 172.21.15.78:0/490703748 shutdown_connections 2024-04-25T01:42:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.738+0000 7fea2b75a640 1 --2- 172.21.15.78:0/490703748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea24103340 0x7fea24109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.739+0000 7fea2b75a640 1 --2- 172.21.15.78:0/490703748 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea24102980 0x7fea24102e00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.739+0000 7fea2b75a640 1 --2- 172.21.15.78:0/490703748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea24101790 0x7fea24101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:52.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.739+0000 7fea2b75a640 1 -- 172.21.15.78:0/490703748 >> 172.21.15.78:0/490703748 conn(0x7fea240fcd20 msgr2=0x7fea240ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.739+0000 7fea2b75a640 1 -- 172.21.15.78:0/490703748 shutdown_connections 2024-04-25T01:42:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.739+0000 7fea2b75a640 1 -- 172.21.15.78:0/490703748 wait complete. 2024-04-25T01:42:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.739+0000 7fea2b75a640 1 Processor -- start 2024-04-25T01:42:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.739+0000 7fea2b75a640 1 -- start start 2024-04-25T01:42:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.739+0000 7fea2b75a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea24101790 0x7fea24197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.739+0000 7fea2b75a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea24102980 0x7fea24197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.739+0000 7fea2b75a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea24103340 0x7fea2419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.739+0000 7fea2b75a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea2410d3f0 con 0x7fea24103340 2024-04-25T01:42:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.739+0000 7fea2b75a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fea2410d270 con 0x7fea24102980 2024-04-25T01:42:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.739+0000 7fea2b75a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea2410d570 con 0x7fea24101790 2024-04-25T01:42:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea29cf7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea24103340 0x7fea2419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea29cf7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea24103340 0x7fea2419f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39152/0 (socket says 172.21.15.78:39152) 2024-04-25T01:42:52.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea29cf7640 1 -- 172.21.15.78:0/2214862844 learned_addr learned my addr 172.21.15.78:0/2214862844 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea294f6640 1 --2- 172.21.15.78:0/2214862844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea24101790 0x7fea24197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea28cf5640 1 --2- 172.21.15.78:0/2214862844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea24102980 0x7fea24197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea29cf7640 1 -- 172.21.15.78:0/2214862844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea24101790 msgr2=0x7fea24197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea29cf7640 1 --2- 172.21.15.78:0/2214862844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea24101790 0x7fea24197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea29cf7640 1 -- 172.21.15.78:0/2214862844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea24102980 msgr2=0x7fea24197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea29cf7640 1 --2- 172.21.15.78:0/2214862844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea24102980 0x7fea24197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea29cf7640 1 -- 172.21.15.78:0/2214862844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2419f6c0 con 0x7fea24103340 2024-04-25T01:42:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea29cf7640 1 --2- 172.21.15.78:0/2214862844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea24103340 0x7fea2419f050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00c970 tx=0x7fea1c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea1a7fc640 1 -- 172.21.15.78:0/2214862844 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c007bb0 con 0x7fea24103340 2024-04-25T01:42:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea1a7fc640 1 -- 172.21.15.78:0/2214862844 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea1c007d50 con 0x7fea24103340 2024-04-25T01:42:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea2b75a640 1 -- 172.21.15.78:0/2214862844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2419f9b0 con 0x7fea24103340 2024-04-25T01:42:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea1a7fc640 1 -- 172.21.15.78:0/2214862844 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c018800 con 0x7fea24103340 2024-04-25T01:42:52.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.740+0000 7fea2b75a640 1 -- 172.21.15.78:0/2214862844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2419ff00 con 0x7fea24103340 2024-04-25T01:42:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.741+0000 7fea2b75a640 1 -- 172.21.15.78:0/2214862844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea24108a50 con 0x7fea24103340 2024-04-25T01:42:52.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.742+0000 7fea1a7fc640 1 -- 172.21.15.78:0/2214862844 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fea1c00fa50 con 0x7fea24103340 2024-04-25T01:42:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.743+0000 7fea1a7fc640 1 --2- 172.21.15.78:0/2214862844 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea100a2c10 0x7fea100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.743+0000 7fea294f6640 1 --2- 172.21.15.78:0/2214862844 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea100a2c10 0x7fea100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.743+0000 7fea1a7fc640 1 -- 172.21.15.78:0/2214862844 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fea1c0cec10 con 0x7fea24103340 2024-04-25T01:42:52.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.744+0000 7fea294f6640 1 --2- 172.21.15.78:0/2214862844 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea100a2c10 0x7fea100a50d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea241027e0 tx=0x7fea20009870 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:52.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.748+0000 7fea1a7fc640 1 -- 172.21.15.78:0/2214862844 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fea1c014030 con 0x7fea24103340 2024-04-25T01:42:52.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.964+0000 7fea2b75a640 1 -- 172.21.15.78:0/2214862844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea24062340 con 0x7fea24103340 2024-04-25T01:42:52.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.964+0000 7fea1a7fc640 1 -- 172.21.15.78:0/2214862844 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7fea1c09c010 con 0x7fea24103340 2024-04-25T01:42:52.954 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:42:52.955 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T01:42:52.955 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 6 2024-04-25T01:42:52.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.966+0000 7fea2b75a640 1 -- 172.21.15.78:0/2214862844 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea100a2c10 msgr2=0x7fea100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:52.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.966+0000 7fea2b75a640 1 --2- 172.21.15.78:0/2214862844 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea100a2c10 0x7fea100a50d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea241027e0 tx=0x7fea20009870 comp rx=0 tx=0).stop 2024-04-25T01:42:52.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.966+0000 7fea2b75a640 1 -- 172.21.15.78:0/2214862844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea24103340 msgr2=0x7fea2419f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:52.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.966+0000 7fea2b75a640 1 --2- 172.21.15.78:0/2214862844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea24103340 0x7fea2419f050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00c970 tx=0x7fea1c00ce30 comp rx=0 tx=0).stop 2024-04-25T01:42:52.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.966+0000 7fea2b75a640 1 -- 172.21.15.78:0/2214862844 shutdown_connections 2024-04-25T01:42:52.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.966+0000 7fea2b75a640 1 --2- 172.21.15.78:0/2214862844 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea100a2c10 0x7fea100a50d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:52.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.966+0000 7fea2b75a640 1 --2- 172.21.15.78:0/2214862844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea24103340 0x7fea2419f050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:52.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.966+0000 7fea2b75a640 1 --2- 172.21.15.78:0/2214862844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea24102980 0x7fea24197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:52.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.966+0000 7fea2b75a640 1 --2- 172.21.15.78:0/2214862844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea24101790 0x7fea24197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:52.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.967+0000 7fea2b75a640 1 -- 172.21.15.78:0/2214862844 >> 172.21.15.78:0/2214862844 conn(0x7fea240fcd20 msgr2=0x7fea24105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:52.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.967+0000 7fea2b75a640 1 -- 172.21.15.78:0/2214862844 shutdown_connections 2024-04-25T01:42:52.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:52.967+0000 7fea2b75a640 1 -- 172.21.15.78:0/2214862844 wait complete. 2024-04-25T01:42:53.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:52 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:53.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:52 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.152:6834/4251210610,v1:172.21.15.152:6835/4251210610] up:boot 2024-04-25T01:42:53.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:52 smithi078 ceph-mon[29469]: fsmap 5 up:standby 2024-04-25T01:42:53.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:52 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T01:42:53.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:52 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:53.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:52 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:53.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:52 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:53.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:52 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:53.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:52 smithi078 ceph-mon[29469]: Deploying daemon mds.f on smithi196 2024-04-25T01:42:53.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:52 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:53.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:52 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.152:6834/4251210610,v1:172.21.15.152:6835/4251210610] up:boot 2024-04-25T01:42:53.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:52 smithi152 ceph-mon[32465]: fsmap 5 up:standby 2024-04-25T01:42:53.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:52 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T01:42:53.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:52 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:53.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:52 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:53.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:52 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:53.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:52 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:53.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:52 smithi152 ceph-mon[32465]: Deploying daemon mds.f on smithi196 2024-04-25T01:42:53.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:52 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:53.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:52 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.152:6834/4251210610,v1:172.21.15.152:6835/4251210610] up:boot 2024-04-25T01:42:53.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:52 smithi196 ceph-mon[32978]: fsmap 5 up:standby 2024-04-25T01:42:53.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:52 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T01:42:53.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:52 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:53.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:52 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:53.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:52 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:53.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:52 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:53.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:52 smithi196 ceph-mon[32978]: Deploying daemon mds.f on smithi196 2024-04-25T01:42:54.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:53 smithi078 ceph-mon[29469]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:54.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:53 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2214862844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:54.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:53 smithi152 ceph-mon[32465]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:54.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:53 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2214862844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:54.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:53 smithi196 ceph-mon[32978]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:54.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:53 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2214862844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:54.365 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:42:54 smithi196 systemd[1]: Starting Ceph mds.f for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:42:54.669 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:42:54 smithi196 podman[63643]: 2024-04-25 01:42:54.303140438 +0000 UTC m=+0.018021798 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:42:54.670 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:42:54 smithi196 podman[63643]: 2024-04-25 01:42:54.428221873 +0000 UTC m=+0.143103272 container create 0bf5b3311815aa67433665a76f09360a6ff9eba037a94378c3cd45e5264cf5f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T01:42:54.670 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:42:54 smithi196 podman[63643]: 2024-04-25 01:42:54.681155717 +0000 UTC m=+0.396037080 container init 0bf5b3311815aa67433665a76f09360a6ff9eba037a94378c3cd45e5264cf5f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T01:42:54.968 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:42:55.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.095+0000 7f878c4b3640 1 -- 172.21.15.78:0/2254537369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784102250 msgr2=0x7f8784108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:55.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.095+0000 7f878c4b3640 1 --2- 172.21.15.78:0/2254537369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784102250 0x7f8784108ae0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f877c009ca0 tx=0x7f877c02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:42:55.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.095+0000 7f878c4b3640 1 -- 172.21.15.78:0/2254537369 shutdown_connections 2024-04-25T01:42:55.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.095+0000 7f878c4b3640 1 --2- 172.21.15.78:0/2254537369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784102250 0x7f8784108ae0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:55.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.095+0000 7f878c4b3640 1 --2- 172.21.15.78:0/2254537369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8784101890 0x7f8784101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:55.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.095+0000 7f878c4b3640 1 --2- 172.21.15.78:0/2254537369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87841006a0 0x7f8784100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:55.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.095+0000 7f878c4b3640 1 -- 172.21.15.78:0/2254537369 >> 172.21.15.78:0/2254537369 conn(0x7f87840fbc30 msgr2=0x7f87840fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:55.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.095+0000 7f878c4b3640 1 -- 172.21.15.78:0/2254537369 shutdown_connections 2024-04-25T01:42:55.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.095+0000 7f878c4b3640 1 -- 172.21.15.78:0/2254537369 wait complete. 2024-04-25T01:42:55.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.096+0000 7f878c4b3640 1 Processor -- start 2024-04-25T01:42:55.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.096+0000 7f878c4b3640 1 -- start start 2024-04-25T01:42:55.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.096+0000 7f878c4b3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87841006a0 0x7f8784104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:55.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.096+0000 7f878c4b3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8784101890 0x7f8784102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:55.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.096+0000 7f878c4b3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784102250 0x7f87841031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:55.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.096+0000 7f878c4b3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f878410c300 con 0x7f8784102250 2024-04-25T01:42:55.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.096+0000 7f878c4b3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f878410c180 con 0x7f87841006a0 2024-04-25T01:42:55.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.096+0000 7f878c4b3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f878410c480 con 0x7f8784101890 2024-04-25T01:42:55.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.096+0000 7f878aa50640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784102250 0x7f87841031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:55.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.096+0000 7f878aa50640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784102250 0x7f87841031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39174/0 (socket says 172.21.15.78:39174) 2024-04-25T01:42:55.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.096+0000 7f878aa50640 1 -- 172.21.15.78:0/3800847815 learned_addr learned my addr 172.21.15.78:0/3800847815 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:55.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.096+0000 7f8789a4e640 1 --2- 172.21.15.78:0/3800847815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8784101890 0x7f8784102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:55.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.096+0000 7f878a24f640 1 --2- 172.21.15.78:0/3800847815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87841006a0 0x7f8784104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:55.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.097+0000 7f878aa50640 1 -- 172.21.15.78:0/3800847815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8784101890 msgr2=0x7f8784102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:55.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.097+0000 7f878aa50640 1 --2- 172.21.15.78:0/3800847815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8784101890 0x7f8784102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:55.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.097+0000 7f878aa50640 1 -- 172.21.15.78:0/3800847815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87841006a0 msgr2=0x7f8784104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:55.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.097+0000 7f878aa50640 1 --2- 172.21.15.78:0/3800847815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87841006a0 0x7f8784104630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:55.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.097+0000 7f878aa50640 1 -- 172.21.15.78:0/3800847815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8784103a40 con 0x7f8784102250 2024-04-25T01:42:55.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.097+0000 7f878aa50640 1 --2- 172.21.15.78:0/3800847815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784102250 0x7f87841031b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f877c031ae0 tx=0x7f877c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:55.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.097+0000 7f877b7fe640 1 -- 172.21.15.78:0/3800847815 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c003b70 con 0x7f8784102250 2024-04-25T01:42:55.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.097+0000 7f877b7fe640 1 -- 172.21.15.78:0/3800847815 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c003d10 con 0x7f8784102250 2024-04-25T01:42:55.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.097+0000 7f878c4b3640 1 -- 172.21.15.78:0/3800847815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87841a1b90 con 0x7f8784102250 2024-04-25T01:42:55.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.097+0000 7f877b7fe640 1 -- 172.21.15.78:0/3800847815 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c02fc40 con 0x7f8784102250 2024-04-25T01:42:55.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.097+0000 7f878c4b3640 1 -- 172.21.15.78:0/3800847815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87841a20e0 con 0x7f8784102250 2024-04-25T01:42:55.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.098+0000 7f878c4b3640 1 -- 172.21.15.78:0/3800847815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87841a22b0 con 0x7f8784102250 2024-04-25T01:42:55.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.100+0000 7f877b7fe640 1 -- 172.21.15.78:0/3800847815 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f877c004050 con 0x7f8784102250 2024-04-25T01:42:55.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.100+0000 7f877b7fe640 1 --2- 172.21.15.78:0/3800847815 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f87700a2b90 0x7f87700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:55.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.100+0000 7f878a24f640 1 --2- 172.21.15.78:0/3800847815 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f87700a2b90 0x7f87700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:55.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.100+0000 7f877b7fe640 1 -- 172.21.15.78:0/3800847815 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f877c0efc20 con 0x7f8784102250 2024-04-25T01:42:55.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.101+0000 7f878a24f640 1 --2- 172.21.15.78:0/3800847815 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f87700a2b90 0x7f87700a5050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f87841016f0 tx=0x7f8780005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:55.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.106+0000 7f877b7fe640 1 -- 172.21.15.78:0/3800847815 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f877c0bd140 con 0x7f8784102250 2024-04-25T01:42:55.114 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:42:54 smithi196 podman[63643]: 2024-04-25 01:42:54.684073959 +0000 UTC m=+0.398955317 container start 0bf5b3311815aa67433665a76f09360a6ff9eba037a94378c3cd45e5264cf5f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T01:42:55.114 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:42:54 smithi196 ceph-mds[63662]: -- 172.21.15.196:0/1275615062 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561811749a40 con 0x56181171cc00 2024-04-25T01:42:55.114 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:42:54 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-f[63658]: starting mds.f at 2024-04-25T01:42:55.114 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:42:54 smithi196 bash[63643]: 0bf5b3311815aa67433665a76f09360a6ff9eba037a94378c3cd45e5264cf5f2 2024-04-25T01:42:55.114 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:42:54 smithi196 systemd[1]: Started Ceph mds.f for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:42:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.321+0000 7f878c4b3640 1 -- 172.21.15.78:0/3800847815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87841a25a0 con 0x7f8784102250 2024-04-25T01:42:55.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.321+0000 7f877b7fe640 1 -- 172.21.15.78:0/3800847815 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7f877c0bca40 con 0x7f8784102250 2024-04-25T01:42:55.311 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:42:55.312 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T01:42:55.312 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 7 2024-04-25T01:42:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.324+0000 7f878c4b3640 1 -- 172.21.15.78:0/3800847815 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f87700a2b90 msgr2=0x7f87700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.324+0000 7f878c4b3640 1 --2- 172.21.15.78:0/3800847815 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f87700a2b90 0x7f87700a5050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f87841016f0 tx=0x7f8780005f90 comp rx=0 tx=0).stop 2024-04-25T01:42:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.324+0000 7f878c4b3640 1 -- 172.21.15.78:0/3800847815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784102250 msgr2=0x7f87841031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.324+0000 7f878c4b3640 1 --2- 172.21.15.78:0/3800847815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784102250 0x7f87841031b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f877c031ae0 tx=0x7f877c031ee0 comp rx=0 tx=0).stop 2024-04-25T01:42:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.324+0000 7f878c4b3640 1 -- 172.21.15.78:0/3800847815 shutdown_connections 2024-04-25T01:42:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.324+0000 7f878c4b3640 1 --2- 172.21.15.78:0/3800847815 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f87700a2b90 0x7f87700a5050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.324+0000 7f878c4b3640 1 --2- 172.21.15.78:0/3800847815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784102250 0x7f87841031b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.324+0000 7f878c4b3640 1 --2- 172.21.15.78:0/3800847815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8784101890 0x7f8784102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.324+0000 7f878c4b3640 1 --2- 172.21.15.78:0/3800847815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87841006a0 0x7f8784104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.324+0000 7f878c4b3640 1 -- 172.21.15.78:0/3800847815 >> 172.21.15.78:0/3800847815 conn(0x7f87840fbc30 msgr2=0x7f87840fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:55.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.324+0000 7f878c4b3640 1 -- 172.21.15.78:0/3800847815 shutdown_connections 2024-04-25T01:42:55.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:55.324+0000 7f878c4b3640 1 -- 172.21.15.78:0/3800847815 wait complete. 2024-04-25T01:42:56.114 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:55 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:56.115 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:55 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:56.115 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:55 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:56.115 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:55 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:56.115 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:55 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:56.115 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:55 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:56.115 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:55 smithi196 ceph-mon[32978]: Deploying daemon mds.c on smithi196 2024-04-25T01:42:56.115 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:55 smithi196 ceph-mon[32978]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:56.115 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:55 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.196:6834/2927651724,v1:172.21.15.196:6835/2927651724] up:boot 2024-04-25T01:42:56.115 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:55 smithi196 ceph-mon[32978]: fsmap 6 up:standby 2024-04-25T01:42:56.116 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:55 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T01:42:56.116 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:55 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3800847815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:56.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:56.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:56.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:56.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:56.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:56.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:56.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:55 smithi078 ceph-mon[29469]: Deploying daemon mds.c on smithi196 2024-04-25T01:42:56.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:55 smithi078 ceph-mon[29469]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:56.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:55 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.196:6834/2927651724,v1:172.21.15.196:6835/2927651724] up:boot 2024-04-25T01:42:56.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:55 smithi078 ceph-mon[29469]: fsmap 6 up:standby 2024-04-25T01:42:56.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T01:42:56.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:55 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3800847815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:56.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:55 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:56.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:55 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:56.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:55 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:56.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:55 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:56.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:55 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:56.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:55 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:56.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:55 smithi152 ceph-mon[32465]: Deploying daemon mds.c on smithi196 2024-04-25T01:42:56.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:55 smithi152 ceph-mon[32465]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:56.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:55 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.196:6834/2927651724,v1:172.21.15.196:6835/2927651724] up:boot 2024-04-25T01:42:56.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:55 smithi152 ceph-mon[32465]: fsmap 6 up:standby 2024-04-25T01:42:56.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:55 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T01:42:56.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:55 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3800847815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:57.115 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:42:57 smithi196 systemd[1]: Starting Ceph mds.c for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:42:57.325 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:42:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.444+0000 7f64077a2640 1 -- 172.21.15.78:0/3076860955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400101870 msgr2=0x7f6400101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.444+0000 7f64077a2640 1 --2- 172.21.15.78:0/3076860955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400101870 0x7f6400101cf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009900 tx=0x7f63f002a410 comp rx=0 tx=0).stop 2024-04-25T01:42:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.444+0000 7f64077a2640 1 -- 172.21.15.78:0/3076860955 shutdown_connections 2024-04-25T01:42:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.444+0000 7f64077a2640 1 --2- 172.21.15.78:0/3076860955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6400102230 0x7f6400108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.444+0000 7f64077a2640 1 --2- 172.21.15.78:0/3076860955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400101870 0x7f6400101cf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f64077a2640 1 --2- 172.21.15.78:0/3076860955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6400100680 0x7f6400100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f64077a2640 1 -- 172.21.15.78:0/3076860955 >> 172.21.15.78:0/3076860955 conn(0x7f64000fbbf0 msgr2=0x7f64000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f64077a2640 1 -- 172.21.15.78:0/3076860955 shutdown_connections 2024-04-25T01:42:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f64077a2640 1 -- 172.21.15.78:0/3076860955 wait complete. 2024-04-25T01:42:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f64077a2640 1 Processor -- start 2024-04-25T01:42:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f64077a2640 1 -- start start 2024-04-25T01:42:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f64077a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400100680 0x7f6400197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f64077a2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6400101870 0x7f6400197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f64077a2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6400102230 0x7f640019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f64077a2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f640010d410 con 0x7f6400100680 2024-04-25T01:42:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f64077a2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f640010d290 con 0x7f6400101870 2024-04-25T01:42:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f64077a2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f640010d590 con 0x7f6400102230 2024-04-25T01:42:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f640553e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400100680 0x7f6400197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f640553e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400100680 0x7f6400197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39192/0 (socket says 172.21.15.78:39192) 2024-04-25T01:42:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f640553e640 1 -- 172.21.15.78:0/1256565272 learned_addr learned my addr 172.21.15.78:0/1256565272 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:57.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.445+0000 7f6405d3f640 1 --2- 172.21.15.78:0/1256565272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6400102230 0x7f640019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:57.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.446+0000 7f6404d3d640 1 --2- 172.21.15.78:0/1256565272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6400101870 0x7f6400197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:57.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.446+0000 7f640553e640 1 -- 172.21.15.78:0/1256565272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6400102230 msgr2=0x7f640019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:57.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.446+0000 7f640553e640 1 --2- 172.21.15.78:0/1256565272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6400102230 0x7f640019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:57.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.446+0000 7f640553e640 1 -- 172.21.15.78:0/1256565272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6400101870 msgr2=0x7f6400197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:57.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.446+0000 7f640553e640 1 --2- 172.21.15.78:0/1256565272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6400101870 0x7f6400197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:57.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.446+0000 7f640553e640 1 -- 172.21.15.78:0/1256565272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640019f680 con 0x7f6400100680 2024-04-25T01:42:57.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.446+0000 7f640553e640 1 --2- 172.21.15.78:0/1256565272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400100680 0x7f6400197a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f63f800d770 tx=0x7f63f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:57.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.446+0000 7f63f67fc640 1 -- 172.21.15.78:0/1256565272 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8017070 con 0x7f6400100680 2024-04-25T01:42:57.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.446+0000 7f63f67fc640 1 -- 172.21.15.78:0/1256565272 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f8009c30 con 0x7f6400100680 2024-04-25T01:42:57.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.446+0000 7f64077a2640 1 -- 172.21.15.78:0/1256565272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640019f970 con 0x7f6400100680 2024-04-25T01:42:57.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.446+0000 7f63f67fc640 1 -- 172.21.15.78:0/1256565272 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8003dd0 con 0x7f6400100680 2024-04-25T01:42:57.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.446+0000 7f64077a2640 1 -- 172.21.15.78:0/1256565272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640019fe90 con 0x7f6400100680 2024-04-25T01:42:57.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.447+0000 7f64077a2640 1 -- 172.21.15.78:0/1256565272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6400107a20 con 0x7f6400100680 2024-04-25T01:42:57.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.448+0000 7f63f67fc640 1 -- 172.21.15.78:0/1256565272 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f63f8003fb0 con 0x7f6400100680 2024-04-25T01:42:57.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.449+0000 7f63f67fc640 1 --2- 172.21.15.78:0/1256565272 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f63ec0a2b90 0x7f63ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:57.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.449+0000 7f6404d3d640 1 --2- 172.21.15.78:0/1256565272 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f63ec0a2b90 0x7f63ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:57.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.449+0000 7f63f67fc640 1 -- 172.21.15.78:0/1256565272 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f63f80ce530 con 0x7f6400100680 2024-04-25T01:42:57.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.449+0000 7f6404d3d640 1 --2- 172.21.15.78:0/1256565272 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f63ec0a2b90 0x7f63ec0a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f63f0000c00 tx=0x7f63f00315d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:57.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.453+0000 7f63f67fc640 1 -- 172.21.15.78:0/1256565272 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f63f809b930 con 0x7f6400100680 2024-04-25T01:42:57.515 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:42:57 smithi196 podman[63951]: 2024-04-25 01:42:57.160119709 +0000 UTC m=+0.017777743 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:42:57.515 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:42:57 smithi196 podman[63951]: 2024-04-25 01:42:57.39985102 +0000 UTC m=+0.257509041 container create 35beadf15ca7257a69719533a0b321384fb6d616a5e6b4cbef7d5e6802b5943d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T01:42:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.667+0000 7f64077a2640 1 -- 172.21.15.78:0/1256565272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f640002d170 con 0x7f6400100680 2024-04-25T01:42:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.668+0000 7f63f67fc640 1 -- 172.21.15.78:0/1256565272 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7f63f8009de0 con 0x7f6400100680 2024-04-25T01:42:57.657 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:42:57.657 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T01:42:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 7 2024-04-25T01:42:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.670+0000 7f64077a2640 1 -- 172.21.15.78:0/1256565272 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f63ec0a2b90 msgr2=0x7f63ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.670+0000 7f64077a2640 1 --2- 172.21.15.78:0/1256565272 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f63ec0a2b90 0x7f63ec0a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f63f0000c00 tx=0x7f63f00315d0 comp rx=0 tx=0).stop 2024-04-25T01:42:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.670+0000 7f64077a2640 1 -- 172.21.15.78:0/1256565272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400100680 msgr2=0x7f6400197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.670+0000 7f64077a2640 1 --2- 172.21.15.78:0/1256565272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400100680 0x7f6400197a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f63f800d770 tx=0x7f63f800dc30 comp rx=0 tx=0).stop 2024-04-25T01:42:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.670+0000 7f64077a2640 1 -- 172.21.15.78:0/1256565272 shutdown_connections 2024-04-25T01:42:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.670+0000 7f64077a2640 1 --2- 172.21.15.78:0/1256565272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6400102230 0x7f640019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:57.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.670+0000 7f64077a2640 1 --2- 172.21.15.78:0/1256565272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6400101870 0x7f6400197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.670+0000 7f64077a2640 1 --2- 172.21.15.78:0/1256565272 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f63ec0a2b90 0x7f63ec0a5050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.670+0000 7f64077a2640 1 --2- 172.21.15.78:0/1256565272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6400100680 0x7f6400197a60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.670+0000 7f64077a2640 1 -- 172.21.15.78:0/1256565272 >> 172.21.15.78:0/1256565272 conn(0x7f64000fbbf0 msgr2=0x7f6400104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.670+0000 7f64077a2640 1 -- 172.21.15.78:0/1256565272 shutdown_connections 2024-04-25T01:42:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:57.670+0000 7f64077a2640 1 -- 172.21.15.78:0/1256565272 wait complete. 2024-04-25T01:42:57.864 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:42:57 smithi196 podman[63951]: 2024-04-25 01:42:57.65271279 +0000 UTC m=+0.510370812 container init 35beadf15ca7257a69719533a0b321384fb6d616a5e6b4cbef7d5e6802b5943d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T01:42:57.864 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:42:57 smithi196 podman[63951]: 2024-04-25 01:42:57.655390792 +0000 UTC m=+0.513048816 container start 35beadf15ca7257a69719533a0b321384fb6d616a5e6b4cbef7d5e6802b5943d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T01:42:57.864 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:42:57 smithi196 ceph-mds[63970]: -- 172.21.15.196:0/3637171736 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5601ede15a40 con 0x5601edde8800 2024-04-25T01:42:57.864 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:42:57 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-c[63966]: starting mds.c at 2024-04-25T01:42:57.864 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:42:57 smithi196 bash[63951]: 35beadf15ca7257a69719533a0b321384fb6d616a5e6b4cbef7d5e6802b5943d 2024-04-25T01:42:57.865 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:42:57 smithi196 systemd[1]: Started Ceph mds.c for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:42:58.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:57 smithi078 ceph-mon[29469]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:58.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:57 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1256565272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:58.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:57 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:58.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:57 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:58.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:57 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:58.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:57 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:58.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:57 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:58.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:57 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:58.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:57 smithi152 ceph-mon[32465]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:58.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:57 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1256565272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:58.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:57 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:58.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:57 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:58.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:57 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:58.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:57 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:58.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:57 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:58.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:57 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:58.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:57 smithi196 ceph-mon[32978]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:42:58.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:57 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1256565272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:42:58.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:57 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:58.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:57 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:58.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:57 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:42:58.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:57 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:42:58.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:57 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:42:58.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:57 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:59.050 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:58 smithi152 ceph-mon[32465]: Deploying daemon mds.h on smithi152 2024-04-25T01:42:59.050 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:58 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.196:6836/1117133390,v1:172.21.15.196:6837/1117133390] up:boot 2024-04-25T01:42:59.050 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:58 smithi152 ceph-mon[32465]: fsmap 7 up:standby 2024-04-25T01:42:59.050 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:58 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T01:42:59.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:58 smithi078 ceph-mon[29469]: Deploying daemon mds.h on smithi152 2024-04-25T01:42:59.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:58 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.196:6836/1117133390,v1:172.21.15.196:6837/1117133390] up:boot 2024-04-25T01:42:59.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:58 smithi078 ceph-mon[29469]: fsmap 7 up:standby 2024-04-25T01:42:59.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:58 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T01:42:59.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:58 smithi196 ceph-mon[32978]: Deploying daemon mds.h on smithi152 2024-04-25T01:42:59.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:58 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.196:6836/1117133390,v1:172.21.15.196:6837/1117133390] up:boot 2024-04-25T01:42:59.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:58 smithi196 ceph-mon[32978]: fsmap 7 up:standby 2024-04-25T01:42:59.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:58 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T01:42:59.671 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:42:59.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.789+0000 7f83ef363640 1 -- 172.21.15.78:0/3893567798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83e8103340 msgr2=0x7f83e8109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:59.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.789+0000 7f83ef363640 1 --2- 172.21.15.78:0/3893567798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83e8103340 0x7f83e8109bd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009ca0 tx=0x7f83e002a8c0 comp rx=0 tx=0).stop 2024-04-25T01:42:59.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.789+0000 7f83ef363640 1 -- 172.21.15.78:0/3893567798 shutdown_connections 2024-04-25T01:42:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.789+0000 7f83ef363640 1 --2- 172.21.15.78:0/3893567798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83e8103340 0x7f83e8109bd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.789+0000 7f83ef363640 1 --2- 172.21.15.78:0/3893567798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83e8102980 0x7f83e8102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.789+0000 7f83ef363640 1 --2- 172.21.15.78:0/3893567798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8101790 0x7f83e8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.789+0000 7f83ef363640 1 -- 172.21.15.78:0/3893567798 >> 172.21.15.78:0/3893567798 conn(0x7f83e80fcd20 msgr2=0x7f83e80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.789+0000 7f83ef363640 1 -- 172.21.15.78:0/3893567798 shutdown_connections 2024-04-25T01:42:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.789+0000 7f83ef363640 1 -- 172.21.15.78:0/3893567798 wait complete. 2024-04-25T01:42:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ef363640 1 Processor -- start 2024-04-25T01:42:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ef363640 1 -- start start 2024-04-25T01:42:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ef363640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8101790 0x7f83e8197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ef363640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83e8102980 0x7f83e8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ef363640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83e8103340 0x7f83e819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ef363640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83e810d410 con 0x7f83e8102980 2024-04-25T01:42:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ef363640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f83e810d290 con 0x7f83e8103340 2024-04-25T01:42:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ef363640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83e810d590 con 0x7f83e8101790 2024-04-25T01:42:59.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ec8fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83e8102980 0x7f83e8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:59.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ec8fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83e8102980 0x7f83e8197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41800/0 (socket says 172.21.15.78:41800) 2024-04-25T01:42:59.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ec8fe640 1 -- 172.21.15.78:0/3020029662 learned_addr learned my addr 172.21.15.78:0/3020029662 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:42:59.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ed0ff640 1 --2- 172.21.15.78:0/3020029662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8101790 0x7f83e8197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:59.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ed900640 1 --2- 172.21.15.78:0/3020029662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83e8103340 0x7f83e819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:59.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ec8fe640 1 -- 172.21.15.78:0/3020029662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8101790 msgr2=0x7f83e8197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:59.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ec8fe640 1 --2- 172.21.15.78:0/3020029662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8101790 0x7f83e8197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:59.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ec8fe640 1 -- 172.21.15.78:0/3020029662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83e8103340 msgr2=0x7f83e819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:59.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ec8fe640 1 --2- 172.21.15.78:0/3020029662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83e8103340 0x7f83e819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:59.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ec8fe640 1 -- 172.21.15.78:0/3020029662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e819f640 con 0x7f83e8102980 2024-04-25T01:42:59.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.790+0000 7f83ec8fe640 1 --2- 172.21.15.78:0/3020029662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83e8102980 0x7f83e8197f60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f83d800e9f0 tx=0x7f83d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:59.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.791+0000 7f83de7fc640 1 -- 172.21.15.78:0/3020029662 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d800cd80 con 0x7f83e8102980 2024-04-25T01:42:59.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.791+0000 7f83de7fc640 1 -- 172.21.15.78:0/3020029662 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d8010910 con 0x7f83e8102980 2024-04-25T01:42:59.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.791+0000 7f83de7fc640 1 -- 172.21.15.78:0/3020029662 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8018990 con 0x7f83e8102980 2024-04-25T01:42:59.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.791+0000 7f83ef363640 1 -- 172.21.15.78:0/3020029662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e819f930 con 0x7f83e8102980 2024-04-25T01:42:59.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.791+0000 7f83ef363640 1 -- 172.21.15.78:0/3020029662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e819fe80 con 0x7f83e8102980 2024-04-25T01:42:59.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.792+0000 7f83ef363640 1 -- 172.21.15.78:0/3020029662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83e8108ab0 con 0x7f83e8102980 2024-04-25T01:42:59.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.793+0000 7f83de7fc640 1 -- 172.21.15.78:0/3020029662 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f83d8010ac0 con 0x7f83e8102980 2024-04-25T01:42:59.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.793+0000 7f83de7fc640 1 --2- 172.21.15.78:0/3020029662 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f83d40a2b90 0x7f83d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:59.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.793+0000 7f83ed0ff640 1 --2- 172.21.15.78:0/3020029662 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f83d40a2b90 0x7f83d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:59.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.793+0000 7f83de7fc640 1 -- 172.21.15.78:0/3020029662 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f83d80cee90 con 0x7f83e8102980 2024-04-25T01:42:59.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.794+0000 7f83ed0ff640 1 --2- 172.21.15.78:0/3020029662 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f83d40a2b90 0x7f83d40a5050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f83e81027e0 tx=0x7f83e4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:59.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:42:59.798+0000 7f83de7fc640 1 -- 172.21.15.78:0/3020029662 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f83d809bc40 con 0x7f83e8102980 2024-04-25T01:42:59.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:00.010+0000 7f83ef363640 1 -- 172.21.15.78:0/3020029662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83e802d390 con 0x7f83e8102980 2024-04-25T01:43:00.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:00.011+0000 7f83de7fc640 1 -- 172.21.15.78:0/3020029662 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f83d809ff40 con 0x7f83e8102980 2024-04-25T01:43:00.000 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:00.000 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T01:43:00.000 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2024-04-25T01:43:00.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:00.013+0000 7f83ef363640 1 -- 172.21.15.78:0/3020029662 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f83d40a2b90 msgr2=0x7f83d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:00.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:00.013+0000 7f83ef363640 1 --2- 172.21.15.78:0/3020029662 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f83d40a2b90 0x7f83d40a5050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f83e81027e0 tx=0x7f83e4005f90 comp rx=0 tx=0).stop 2024-04-25T01:43:00.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:00.013+0000 7f83ef363640 1 -- 172.21.15.78:0/3020029662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83e8102980 msgr2=0x7f83e8197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:00.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:00.013+0000 7f83ef363640 1 --2- 172.21.15.78:0/3020029662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83e8102980 0x7f83e8197f60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f83d800e9f0 tx=0x7f83d800eeb0 comp rx=0 tx=0).stop 2024-04-25T01:43:00.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:00.013+0000 7f83ef363640 1 -- 172.21.15.78:0/3020029662 shutdown_connections 2024-04-25T01:43:00.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:00.013+0000 7f83ef363640 1 --2- 172.21.15.78:0/3020029662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83e8103340 0x7f83e819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:00.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:00.013+0000 7f83ef363640 1 --2- 172.21.15.78:0/3020029662 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f83d40a2b90 0x7f83d40a5050 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:00.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:00.013+0000 7f83ef363640 1 --2- 172.21.15.78:0/3020029662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83e8102980 0x7f83e8197f60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:00.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:00.013+0000 7f83ef363640 1 --2- 172.21.15.78:0/3020029662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83e8101790 0x7f83e8197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:00.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:00.013+0000 7f83ef363640 1 -- 172.21.15.78:0/3020029662 >> 172.21.15.78:0/3020029662 conn(0x7f83e80fcd20 msgr2=0x7f83e8105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:00.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:00.013+0000 7f83ef363640 1 -- 172.21.15.78:0/3020029662 shutdown_connections 2024-04-25T01:43:00.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:00.013+0000 7f83ef363640 1 -- 172.21.15.78:0/3020029662 wait complete. 2024-04-25T01:43:00.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:42:59 smithi078 ceph-mon[29469]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:00.292 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:42:59 smithi152 ceph-mon[32465]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:00.292 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:43:00 smithi152 systemd[1]: Starting Ceph mds.h for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:43:00.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:42:59 smithi196 ceph-mon[32978]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:00.546 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:43:00 smithi152 podman[62484]: 2024-04-25 01:43:00.205876944 +0000 UTC m=+0.018784652 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:43:00.546 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:43:00 smithi152 podman[62484]: 2024-04-25 01:43:00.31915413 +0000 UTC m=+0.132061811 container create b6b3e0c9d0eef445ef3cbe466aa67c34fa749b5e849c394a1cacce7b507a1ef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:43:00.944 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:43:00 smithi152 podman[62484]: 2024-04-25 01:43:00.592639212 +0000 UTC m=+0.405546892 container init b6b3e0c9d0eef445ef3cbe466aa67c34fa749b5e849c394a1cacce7b507a1ef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:43:00.944 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:43:00 smithi152 podman[62484]: 2024-04-25 01:43:00.595164835 +0000 UTC m=+0.408072516 container start b6b3e0c9d0eef445ef3cbe466aa67c34fa749b5e849c394a1cacce7b507a1ef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T01:43:00.944 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:43:00 smithi152 ceph-mds[62505]: -- 172.21.15.152:0/2694457468 <== mon.2 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f1f2dba40 con 0x560f1f2af000 2024-04-25T01:43:00.944 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:43:00 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-h[62501]: starting mds.h at 2024-04-25T01:43:00.944 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:43:00 smithi152 bash[62484]: b6b3e0c9d0eef445ef3cbe466aa67c34fa749b5e849c394a1cacce7b507a1ef8 2024-04-25T01:43:00.944 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:43:00 smithi152 systemd[1]: Started Ceph mds.h for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:43:01.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:00 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3020029662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:01.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:00 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:01.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:00 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:01.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:00 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:01.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:00 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:43:01.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:00 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:43:01.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:00 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:01.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:00 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3020029662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:01.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:00 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:01.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:00 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:01.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:00 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:01.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:00 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:43:01.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:00 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:43:01.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:00 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:01.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:00 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3020029662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:01.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:00 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:01.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:00 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:01.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:00 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:01.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:00 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:43:01.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:00 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:43:01.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:00 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:02.013 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:43:02.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.140+0000 7fc28e27b640 1 -- 172.21.15.78:0/4230096054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc288100690 msgr2=0x7fc288100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:02.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.140+0000 7fc28e27b640 1 --2- 172.21.15.78:0/4230096054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc288100690 0x7fc288100ab0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc278009900 tx=0x7fc27802a410 comp rx=0 tx=0).stop 2024-04-25T01:43:02.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.140+0000 7fc28e27b640 1 -- 172.21.15.78:0/4230096054 shutdown_connections 2024-04-25T01:43:02.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.140+0000 7fc28e27b640 1 --2- 172.21.15.78:0/4230096054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc288102240 0x7fc288108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:02.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.140+0000 7fc28e27b640 1 --2- 172.21.15.78:0/4230096054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc288101880 0x7fc288101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:02.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.140+0000 7fc28e27b640 1 --2- 172.21.15.78:0/4230096054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc288100690 0x7fc288100ab0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:02.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.140+0000 7fc28e27b640 1 -- 172.21.15.78:0/4230096054 >> 172.21.15.78:0/4230096054 conn(0x7fc2880fbc20 msgr2=0x7fc2880fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:02.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.140+0000 7fc28e27b640 1 -- 172.21.15.78:0/4230096054 shutdown_connections 2024-04-25T01:43:02.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.140+0000 7fc28e27b640 1 -- 172.21.15.78:0/4230096054 wait complete. 2024-04-25T01:43:02.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.140+0000 7fc28e27b640 1 Processor -- start 2024-04-25T01:43:02.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.140+0000 7fc28e27b640 1 -- start start 2024-04-25T01:43:02.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc28e27b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc288100690 0x7fc28806a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:02.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc28e27b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc288101880 0x7fc28819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:02.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc28e27b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc288102240 0x7fc28819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:02.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc28e27b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc28810c310 con 0x7fc288100690 2024-04-25T01:43:02.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc28e27b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc28810c190 con 0x7fc288101880 2024-04-25T01:43:02.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc28e27b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc28810c490 con 0x7fc288102240 2024-04-25T01:43:02.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc287fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc288100690 0x7fc28806a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:02.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc287fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc288100690 0x7fc28806a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41816/0 (socket says 172.21.15.78:41816) 2024-04-25T01:43:02.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc287fff640 1 -- 172.21.15.78:0/3669885319 learned_addr learned my addr 172.21.15.78:0/3669885319 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:02.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc28c818640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc288102240 0x7fc28819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:02.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc2877fe640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc288101880 0x7fc28819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:02.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc287fff640 1 -- 172.21.15.78:0/3669885319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc288102240 msgr2=0x7fc28819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:02.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc287fff640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc288102240 0x7fc28819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:02.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc287fff640 1 -- 172.21.15.78:0/3669885319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc288101880 msgr2=0x7fc28819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:02.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc287fff640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc288101880 0x7fc28819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:02.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc287fff640 1 -- 172.21.15.78:0/3669885319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28819f6d0 con 0x7fc288100690 2024-04-25T01:43:02.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.141+0000 7fc2877fe640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc288101880 0x7fc28819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:43:02.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.142+0000 7fc287fff640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc288100690 0x7fc28806a970 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc278031610 tx=0x7fc278004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:02.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.142+0000 7fc2857fa640 1 -- 172.21.15.78:0/3669885319 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27803f030 con 0x7fc288100690 2024-04-25T01:43:02.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.142+0000 7fc28e27b640 1 -- 172.21.15.78:0/3669885319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28819f960 con 0x7fc288100690 2024-04-25T01:43:02.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.142+0000 7fc2857fa640 1 -- 172.21.15.78:0/3669885319 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc278003ad0 con 0x7fc288100690 2024-04-25T01:43:02.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.142+0000 7fc2857fa640 1 -- 172.21.15.78:0/3669885319 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278035070 con 0x7fc288100690 2024-04-25T01:43:02.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.142+0000 7fc28e27b640 1 -- 172.21.15.78:0/3669885319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28819fe50 con 0x7fc288100690 2024-04-25T01:43:02.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.143+0000 7fc28e27b640 1 -- 172.21.15.78:0/3669885319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2881079b0 con 0x7fc288100690 2024-04-25T01:43:02.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.144+0000 7fc2857fa640 1 -- 172.21.15.78:0/3669885319 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc278042030 con 0x7fc288100690 2024-04-25T01:43:02.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.145+0000 7fc2857fa640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc2700a2b90 0x7fc2700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:02.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.145+0000 7fc2877fe640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc2700a2b90 0x7fc2700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:02.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.145+0000 7fc2857fa640 1 -- 172.21.15.78:0/3669885319 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc2780ef380 con 0x7fc288100690 2024-04-25T01:43:02.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.146+0000 7fc2877fe640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc2700a2b90 0x7fc2700a5050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc27c005d00 tx=0x7fc27c005ed0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:02.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.150+0000 7fc2857fa640 1 -- 172.21.15.78:0/3669885319 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc2780bc780 con 0x7fc288100690 2024-04-25T01:43:02.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:01 smithi078 ceph-mon[29469]: Deploying daemon mds.i on smithi196 2024-04-25T01:43:02.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:01 smithi078 ceph-mon[29469]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:02.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:01 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] up:boot 2024-04-25T01:43:02.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:01 smithi078 ceph-mon[29469]: fsmap 8 up:standby 2024-04-25T01:43:02.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:01 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T01:43:02.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:01 smithi152 ceph-mon[32465]: Deploying daemon mds.i on smithi196 2024-04-25T01:43:02.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:01 smithi152 ceph-mon[32465]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:02.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:01 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] up:boot 2024-04-25T01:43:02.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:01 smithi152 ceph-mon[32465]: fsmap 8 up:standby 2024-04-25T01:43:02.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:01 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T01:43:02.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.365+0000 7fc28e27b640 1 -- 172.21.15.78:0/3669885319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc28802d170 con 0x7fc288100690 2024-04-25T01:43:02.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.366+0000 7fc2857fa640 1 -- 172.21.15.78:0/3669885319 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7fc2780bc080 con 0x7fc288100690 2024-04-25T01:43:02.354 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:02.355 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T01:43:02.355 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 9 2024-04-25T01:43:02.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.367+0000 7fc28e27b640 1 -- 172.21.15.78:0/3669885319 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc2700a2b90 msgr2=0x7fc2700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.367+0000 7fc28e27b640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc2700a2b90 0x7fc2700a5050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc27c005d00 tx=0x7fc27c005ed0 comp rx=0 tx=0).stop 2024-04-25T01:43:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.367+0000 7fc28e27b640 1 -- 172.21.15.78:0/3669885319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc288100690 msgr2=0x7fc28806a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.367+0000 7fc28e27b640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc288100690 0x7fc28806a970 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc278031610 tx=0x7fc278004270 comp rx=0 tx=0).stop 2024-04-25T01:43:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.368+0000 7fc28e27b640 1 -- 172.21.15.78:0/3669885319 shutdown_connections 2024-04-25T01:43:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.368+0000 7fc28e27b640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc288102240 0x7fc28819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.368+0000 7fc28e27b640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc288101880 0x7fc28819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.368+0000 7fc28e27b640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc2700a2b90 0x7fc2700a5050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.368+0000 7fc28e27b640 1 --2- 172.21.15.78:0/3669885319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc288100690 0x7fc28806a970 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.368+0000 7fc28e27b640 1 -- 172.21.15.78:0/3669885319 >> 172.21.15.78:0/3669885319 conn(0x7fc2880fbc20 msgr2=0x7fc288104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.368+0000 7fc28e27b640 1 -- 172.21.15.78:0/3669885319 shutdown_connections 2024-04-25T01:43:02.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:02.368+0000 7fc28e27b640 1 -- 172.21.15.78:0/3669885319 wait complete. 2024-04-25T01:43:02.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:01 smithi196 ceph-mon[32978]: Deploying daemon mds.i on smithi196 2024-04-25T01:43:02.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:01 smithi196 ceph-mon[32978]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:02.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:01 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] up:boot 2024-04-25T01:43:02.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:01 smithi196 ceph-mon[32978]: fsmap 8 up:standby 2024-04-25T01:43:02.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:01 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T01:43:02.961 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:43:02 smithi196 systemd[1]: Starting Ceph mds.i for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:43:03.228 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:02 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3669885319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:03.228 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:43:03 smithi196 podman[64268]: 2024-04-25 01:43:02.992604932 +0000 UTC m=+0.018123105 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:43:03.229 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:43:03 smithi196 podman[64268]: 2024-04-25 01:43:03.096445415 +0000 UTC m=+0.121963564 container create 3241e6bf050a2c9d4a62fb33aaf745011eb38ec7c5d88386da4160a70d568a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:43:03.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:02 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3669885319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:03.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:02 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3669885319' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:03.614 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:43:03 smithi196 podman[64268]: 2024-04-25 01:43:03.349219049 +0000 UTC m=+0.374737197 container init 3241e6bf050a2c9d4a62fb33aaf745011eb38ec7c5d88386da4160a70d568a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T01:43:03.614 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:43:03 smithi196 podman[64268]: 2024-04-25 01:43:03.351991754 +0000 UTC m=+0.377509905 container start 3241e6bf050a2c9d4a62fb33aaf745011eb38ec7c5d88386da4160a70d568a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T01:43:03.614 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:43:03 smithi196 ceph-mds[64287]: -- 172.21.15.196:0/2218480637 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e890c0ba40 con 0x55e890bde800 2024-04-25T01:43:03.615 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:43:03 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-i[64283]: starting mds.i at 2024-04-25T01:43:03.615 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:43:03 smithi196 bash[64268]: 3241e6bf050a2c9d4a62fb33aaf745011eb38ec7c5d88386da4160a70d568a2a 2024-04-25T01:43:03.615 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:43:03 smithi196 systemd[1]: Started Ceph mds.i for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:43:04.221 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:03 smithi078 ceph-mon[29469]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:04.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:04.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:04.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:04.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:43:04.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:43:04.222 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:03 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:04.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:03 smithi152 ceph-mon[32465]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:04.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:04.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:04.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:04.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:43:04.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:43:04.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:03 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:04.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:03 smithi196 ceph-mon[32978]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:04.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:04.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:04.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:04.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:43:04.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:43:04.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:03 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:04.368 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:43:04.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.479+0000 7f9011ee1640 1 -- 172.21.15.78:0/665136688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f900c102210 msgr2=0x7f900c104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:04.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.479+0000 7f9011ee1640 1 --2- 172.21.15.78:0/665136688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f900c102210 0x7f900c104600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009900 tx=0x7f8ffc02a410 comp rx=0 tx=0).stop 2024-04-25T01:43:04.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.479+0000 7f9011ee1640 1 -- 172.21.15.78:0/665136688 shutdown_connections 2024-04-25T01:43:04.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.479+0000 7f9011ee1640 1 --2- 172.21.15.78:0/665136688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f900c107470 0x7f900c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:04.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.479+0000 7f9011ee1640 1 --2- 172.21.15.78:0/665136688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900c104b40 0x7f900c106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:04.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.479+0000 7f9011ee1640 1 --2- 172.21.15.78:0/665136688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f900c102210 0x7f900c104600 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:04.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.479+0000 7f9011ee1640 1 -- 172.21.15.78:0/665136688 >> 172.21.15.78:0/665136688 conn(0x7f900c0fbc20 msgr2=0x7f900c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:04.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.479+0000 7f9011ee1640 1 -- 172.21.15.78:0/665136688 shutdown_connections 2024-04-25T01:43:04.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.479+0000 7f9011ee1640 1 -- 172.21.15.78:0/665136688 wait complete. 2024-04-25T01:43:04.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.479+0000 7f9011ee1640 1 Processor -- start 2024-04-25T01:43:04.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f9011ee1640 1 -- start start 2024-04-25T01:43:04.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f9011ee1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f900c102210 0x7f900c101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:04.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f9011ee1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900c104b40 0x7f900c1001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:04.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f9011ee1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f900c107470 0x7f900c100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:04.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f9011ee1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f900c10d490 con 0x7f900c107470 2024-04-25T01:43:04.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f9011ee1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f900c10d310 con 0x7f900c104b40 2024-04-25T01:43:04.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f9011ee1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f900c10d610 con 0x7f900c102210 2024-04-25T01:43:04.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f900bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f900c107470 0x7f900c100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:04.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f900bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f900c107470 0x7f900c100700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41836/0 (socket says 172.21.15.78:41836) 2024-04-25T01:43:04.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f900bfff640 1 -- 172.21.15.78:0/662495705 learned_addr learned my addr 172.21.15.78:0/662495705 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:04.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f900b7fe640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f900c102210 0x7f900c101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:04.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f900bfff640 1 -- 172.21.15.78:0/662495705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f900c102210 msgr2=0x7f900c101b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:04.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f900affd640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900c104b40 0x7f900c1001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:04.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f900bfff640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f900c102210 0x7f900c101b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:04.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f900bfff640 1 -- 172.21.15.78:0/662495705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900c104b40 msgr2=0x7f900c1001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:04.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f900bfff640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900c104b40 0x7f900c1001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:04.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f900bfff640 1 -- 172.21.15.78:0/662495705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c100f90 con 0x7f900c107470 2024-04-25T01:43:04.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f900affd640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900c104b40 0x7f900c1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:43:04.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f900bfff640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f900c107470 0x7f900c100700 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f900400cce0 tx=0x7f900400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:04.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f9008ff9640 1 -- 172.21.15.78:0/662495705 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900400f450 con 0x7f900c107470 2024-04-25T01:43:04.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f9011ee1640 1 -- 172.21.15.78:0/662495705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c1a1bc0 con 0x7f900c107470 2024-04-25T01:43:04.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f9008ff9640 1 -- 172.21.15.78:0/662495705 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f900400fad0 con 0x7f900c107470 2024-04-25T01:43:04.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f9008ff9640 1 -- 172.21.15.78:0/662495705 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004018c10 con 0x7f900c107470 2024-04-25T01:43:04.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.480+0000 7f9011ee1640 1 -- 172.21.15.78:0/662495705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c1a2110 con 0x7f900c107470 2024-04-25T01:43:04.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.481+0000 7f9011ee1640 1 -- 172.21.15.78:0/662495705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c0fd710 con 0x7f900c107470 2024-04-25T01:43:04.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.483+0000 7f9008ff9640 1 -- 172.21.15.78:0/662495705 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f900400f5f0 con 0x7f900c107470 2024-04-25T01:43:04.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.483+0000 7f9008ff9640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8ff40a2b90 0x7f8ff40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:04.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.483+0000 7f900b7fe640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8ff40a2b90 0x7f8ff40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:04.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.483+0000 7f9008ff9640 1 -- 172.21.15.78:0/662495705 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f90040cf3f0 con 0x7f900c107470 2024-04-25T01:43:04.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.483+0000 7f900b7fe640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8ff40a2b90 0x7f8ff40a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc031610 tx=0x7f8ffc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:04.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.487+0000 7f9008ff9640 1 -- 172.21.15.78:0/662495705 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9004014030 con 0x7f900c107470 2024-04-25T01:43:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.706+0000 7f9011ee1640 1 -- 172.21.15.78:0/662495705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f900c02d170 con 0x7f900c107470 2024-04-25T01:43:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.707+0000 7f9008ff9640 1 -- 172.21.15.78:0/662495705 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7f900409c860 con 0x7f900c107470 2024-04-25T01:43:04.695 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:04.696 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T01:43:04.696 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 10 2024-04-25T01:43:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.708+0000 7f9011ee1640 1 -- 172.21.15.78:0/662495705 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8ff40a2b90 msgr2=0x7f8ff40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.708+0000 7f9011ee1640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8ff40a2b90 0x7f8ff40a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc031610 tx=0x7f8ffc000bc0 comp rx=0 tx=0).stop 2024-04-25T01:43:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.708+0000 7f9011ee1640 1 -- 172.21.15.78:0/662495705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f900c107470 msgr2=0x7f900c100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.708+0000 7f9011ee1640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f900c107470 0x7f900c100700 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f900400cce0 tx=0x7f900400a620 comp rx=0 tx=0).stop 2024-04-25T01:43:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.709+0000 7f9011ee1640 1 -- 172.21.15.78:0/662495705 shutdown_connections 2024-04-25T01:43:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.709+0000 7f9011ee1640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8ff40a2b90 0x7f8ff40a5050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.709+0000 7f9011ee1640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f900c107470 0x7f900c100700 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.709+0000 7f9011ee1640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f900c104b40 0x7f900c1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.709+0000 7f9011ee1640 1 --2- 172.21.15.78:0/662495705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f900c102210 0x7f900c101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.709+0000 7f9011ee1640 1 -- 172.21.15.78:0/662495705 >> 172.21.15.78:0/662495705 conn(0x7f900c0fbc20 msgr2=0x7f900c102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.709+0000 7f9011ee1640 1 -- 172.21.15.78:0/662495705 shutdown_connections 2024-04-25T01:43:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:04.709+0000 7f9011ee1640 1 -- 172.21.15.78:0/662495705 wait complete. 2024-04-25T01:43:05.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:04 smithi078 ceph-mon[29469]: Deploying daemon mds.g on smithi078 2024-04-25T01:43:05.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:04 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] up:boot 2024-04-25T01:43:05.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:04 smithi078 ceph-mon[29469]: fsmap 9 up:standby 2024-04-25T01:43:05.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:04 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T01:43:05.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:04 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/662495705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:05.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:04 smithi152 ceph-mon[32465]: Deploying daemon mds.g on smithi078 2024-04-25T01:43:05.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:04 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] up:boot 2024-04-25T01:43:05.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:04 smithi152 ceph-mon[32465]: fsmap 9 up:standby 2024-04-25T01:43:05.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:04 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T01:43:05.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:04 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/662495705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:05.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:04 smithi196 ceph-mon[32978]: Deploying daemon mds.g on smithi078 2024-04-25T01:43:05.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:04 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] up:boot 2024-04-25T01:43:05.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:04 smithi196 ceph-mon[32978]: fsmap 9 up:standby 2024-04-25T01:43:05.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:04 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T01:43:05.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:04 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/662495705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:05.954 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:43:05 smithi078 systemd[1]: Starting Ceph mds.g for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:43:06.252 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:05 smithi078 ceph-mon[29469]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:06.252 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:43:05 smithi078 podman[68093]: 2024-04-25 01:43:05.867625817 +0000 UTC m=+0.017606224 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:43:06.252 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:43:05 smithi078 podman[68093]: 2024-04-25 01:43:05.998902165 +0000 UTC m=+0.148882571 container create e27d76e841e0f09dfc17160d2fe938339767fbdecd02f0eb2d6e91caf7865e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:43:06.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:05 smithi152 ceph-mon[32465]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:06.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:05 smithi196 ceph-mon[32978]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:06.505 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:43:06 smithi078 podman[68093]: 2024-04-25 01:43:06.263192652 +0000 UTC m=+0.413173059 container init e27d76e841e0f09dfc17160d2fe938339767fbdecd02f0eb2d6e91caf7865e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T01:43:06.505 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:43:06 smithi078 podman[68093]: 2024-04-25 01:43:06.265986614 +0000 UTC m=+0.415967027 container start e27d76e841e0f09dfc17160d2fe938339767fbdecd02f0eb2d6e91caf7865e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-g, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:43:06.505 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:43:06 smithi078 ceph-mds[68112]: -- 172.21.15.78:0/775887917 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556bc526da40 con 0x556bc5241000 2024-04-25T01:43:06.505 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:43:06 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-g[68108]: starting mds.g at 2024-04-25T01:43:06.505 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:43:06 smithi078 bash[68093]: e27d76e841e0f09dfc17160d2fe938339767fbdecd02f0eb2d6e91caf7865e5b 2024-04-25T01:43:06.505 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:43:06 smithi078 systemd[1]: Started Ceph mds.g for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:43:06.709 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:43:06.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.822+0000 7f2842e1c640 1 -- 172.21.15.78:0/1315349055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f283c102200 msgr2=0x7f283c1045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:06.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.822+0000 7f2842e1c640 1 --2- 172.21.15.78:0/1315349055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f283c102200 0x7f283c1045f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2830009900 tx=0x7f283002a410 comp rx=0 tx=0).stop 2024-04-25T01:43:06.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 -- 172.21.15.78:0/1315349055 shutdown_connections 2024-04-25T01:43:06.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 --2- 172.21.15.78:0/1315349055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f283c107460 0x7f283c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:06.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 --2- 172.21.15.78:0/1315349055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f283c104b30 0x7f283c106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:06.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 --2- 172.21.15.78:0/1315349055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f283c102200 0x7f283c1045f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:06.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 -- 172.21.15.78:0/1315349055 >> 172.21.15.78:0/1315349055 conn(0x7f283c0fbbf0 msgr2=0x7f283c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:06.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 -- 172.21.15.78:0/1315349055 shutdown_connections 2024-04-25T01:43:06.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 -- 172.21.15.78:0/1315349055 wait complete. 2024-04-25T01:43:06.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 Processor -- start 2024-04-25T01:43:06.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 -- start start 2024-04-25T01:43:06.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f283c102200 0x7f283c101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:06.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f283c104b30 0x7f283c1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:06.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f283c107460 0x7f283c1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:06.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f283c10d420 con 0x7f283c104b30 2024-04-25T01:43:06.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f283c10d2a0 con 0x7f283c107460 2024-04-25T01:43:06.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f2842e1c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f283c10d5a0 con 0x7f283c102200 2024-04-25T01:43:06.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f283bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f283c104b30 0x7f283c1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:06.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.823+0000 7f283bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f283c104b30 0x7f283c1001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41876/0 (socket says 172.21.15.78:41876) 2024-04-25T01:43:06.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f283bfff640 1 -- 172.21.15.78:0/2080000153 learned_addr learned my addr 172.21.15.78:0/2080000153 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:06.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f2840bb8640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f283c102200 0x7f283c101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:06.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f28413b9640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f283c107460 0x7f283c1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:06.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f283bfff640 1 -- 172.21.15.78:0/2080000153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f283c102200 msgr2=0x7f283c101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:06.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f283bfff640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f283c102200 0x7f283c101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:06.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f283bfff640 1 -- 172.21.15.78:0/2080000153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f283c107460 msgr2=0x7f283c1006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:06.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f283bfff640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f283c107460 0x7f283c1006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:06.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f283bfff640 1 -- 172.21.15.78:0/2080000153 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c100f80 con 0x7f283c104b30 2024-04-25T01:43:06.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f28413b9640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f283c107460 0x7f283c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:06.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f283bfff640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f283c104b30 0x7f283c1001b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f282800c970 tx=0x7f282800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:06.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f2839ffb640 1 -- 172.21.15.78:0/2080000153 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828007bb0 con 0x7f283c104b30 2024-04-25T01:43:06.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f2842e1c640 1 -- 172.21.15.78:0/2080000153 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283c1a1bc0 con 0x7f283c104b30 2024-04-25T01:43:06.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f2839ffb640 1 -- 172.21.15.78:0/2080000153 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2828007d50 con 0x7f283c104b30 2024-04-25T01:43:06.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f2839ffb640 1 -- 172.21.15.78:0/2080000153 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282800f450 con 0x7f283c104b30 2024-04-25T01:43:06.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.824+0000 7f2842e1c640 1 -- 172.21.15.78:0/2080000153 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283c1a2110 con 0x7f283c104b30 2024-04-25T01:43:06.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.825+0000 7f2842e1c640 1 -- 172.21.15.78:0/2080000153 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f283c0fd6e0 con 0x7f283c104b30 2024-04-25T01:43:06.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.827+0000 7f2839ffb640 1 -- 172.21.15.78:0/2080000153 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f28280046c0 con 0x7f283c104b30 2024-04-25T01:43:06.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.828+0000 7f2839ffb640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f282c0a2b90 0x7f282c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:06.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.828+0000 7f2839ffb640 1 -- 172.21.15.78:0/2080000153 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f28280cebf0 con 0x7f283c104b30 2024-04-25T01:43:06.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.828+0000 7f2840bb8640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f282c0a2b90 0x7f282c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:06.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.828+0000 7f2840bb8640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f282c0a2b90 0x7f282c0a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f28300314e0 tx=0x7f2830000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:06.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:06.832+0000 7f2839ffb640 1 -- 172.21.15.78:0/2080000153 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2828014030 con 0x7f283c104b30 2024-04-25T01:43:07.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:07.070+0000 7f2842e1c640 1 -- 172.21.15.78:0/2080000153 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f283c02d370 con 0x7f283c104b30 2024-04-25T01:43:07.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:07.071+0000 7f2839ffb640 1 -- 172.21.15.78:0/2080000153 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7f282809bff0 con 0x7f283c104b30 2024-04-25T01:43:07.060 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:07.060 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/1337519666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1337519666},{"type":"v1","addr":"172.21.15.78:6839","nonce":1337519666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T01:43:07.061 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 11 2024-04-25T01:43:07.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:07.073+0000 7f2842e1c640 1 -- 172.21.15.78:0/2080000153 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f282c0a2b90 msgr2=0x7f282c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:07.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:07.073+0000 7f2842e1c640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f282c0a2b90 0x7f282c0a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f28300314e0 tx=0x7f2830000bc0 comp rx=0 tx=0).stop 2024-04-25T01:43:07.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:07.073+0000 7f2842e1c640 1 -- 172.21.15.78:0/2080000153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f283c104b30 msgr2=0x7f283c1001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:07.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:07.073+0000 7f2842e1c640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f283c104b30 0x7f283c1001b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f282800c970 tx=0x7f282800ce30 comp rx=0 tx=0).stop 2024-04-25T01:43:07.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:07.073+0000 7f2842e1c640 1 -- 172.21.15.78:0/2080000153 shutdown_connections 2024-04-25T01:43:07.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:07.073+0000 7f2842e1c640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f283c107460 0x7f283c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:07.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:07.073+0000 7f2842e1c640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f282c0a2b90 0x7f282c0a5050 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:07.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:07.073+0000 7f2842e1c640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f283c104b30 0x7f283c1001b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:07.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:07.073+0000 7f2842e1c640 1 --2- 172.21.15.78:0/2080000153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f283c102200 0x7f283c101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:07.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:07.073+0000 7f2842e1c640 1 -- 172.21.15.78:0/2080000153 >> 172.21.15.78:0/2080000153 conn(0x7f283c0fbbf0 msgr2=0x7f283c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:07.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:07.073+0000 7f2842e1c640 1 -- 172.21.15.78:0/2080000153 shutdown_connections 2024-04-25T01:43:07.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:07.073+0000 7f2842e1c640 1 -- 172.21.15.78:0/2080000153 wait complete. 2024-04-25T01:43:07.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:07 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:07.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:07 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:07.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:07 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:07.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:07 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:43:07.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:07 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:43:07.752 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:07 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:07.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:07 smithi078 ceph-mon[29469]: Deploying daemon mds.a on smithi078 2024-04-25T01:43:07.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:07 smithi078 ceph-mon[29469]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:07.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:07 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.78:6838/1337519666,v1:172.21.15.78:6839/1337519666] up:boot 2024-04-25T01:43:07.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:07 smithi078 ceph-mon[29469]: fsmap 10 up:standby 2024-04-25T01:43:07.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:07 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T01:43:07.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:07 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2080000153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:07.796 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:07 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:07.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:07 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:07.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:07 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:07.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:07 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:43:07.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:07 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:43:07.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:07 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:07.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:07 smithi152 ceph-mon[32465]: Deploying daemon mds.a on smithi078 2024-04-25T01:43:07.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:07 smithi152 ceph-mon[32465]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:07.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:07 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.78:6838/1337519666,v1:172.21.15.78:6839/1337519666] up:boot 2024-04-25T01:43:07.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:07 smithi152 ceph-mon[32465]: fsmap 10 up:standby 2024-04-25T01:43:07.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:07 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T01:43:07.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:07 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2080000153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:07.864 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:07 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:07.864 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:07 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:07.864 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:07 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:07.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:07 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:43:07.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:07 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:43:07.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:07 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:07.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:07 smithi196 ceph-mon[32978]: Deploying daemon mds.a on smithi078 2024-04-25T01:43:07.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:07 smithi196 ceph-mon[32978]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:07.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:07 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.78:6838/1337519666,v1:172.21.15.78:6839/1337519666] up:boot 2024-04-25T01:43:07.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:07 smithi196 ceph-mon[32978]: fsmap 10 up:standby 2024-04-25T01:43:07.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:07 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T01:43:07.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:07 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2080000153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:08.755 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:43:08 smithi078 systemd[1]: Starting Ceph mds.a for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:43:09.051 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:43:08 smithi078 podman[68452]: 2024-04-25 01:43:08.79129573 +0000 UTC m=+0.111774488 container create b1e800d9d8d9faa251a79571bd4c857502958ffc28cc7f79ae3a123a873dadd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T01:43:09.051 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:43:08 smithi078 podman[68452]: 2024-04-25 01:43:08.697450932 +0000 UTC m=+0.017929660 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:43:09.051 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:43:09 smithi078 podman[68452]: 2024-04-25 01:43:09.035577234 +0000 UTC m=+0.356055950 container init b1e800d9d8d9faa251a79571bd4c857502958ffc28cc7f79ae3a123a873dadd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T01:43:09.051 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:43:09 smithi078 podman[68452]: 2024-04-25 01:43:09.038435742 +0000 UTC m=+0.358914464 container start b1e800d9d8d9faa251a79571bd4c857502958ffc28cc7f79ae3a123a873dadd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:43:09.074 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:43:09.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.189+0000 7f20b05c1640 1 -- 172.21.15.78:0/1214269463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a81074e0 msgr2=0x7f20a81098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.189+0000 7f20b05c1640 1 --2- 172.21.15.78:0/1214269463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a81074e0 0x7f20a81098d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009ca0 tx=0x7f20a002a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 -- 172.21.15.78:0/1214269463 shutdown_connections 2024-04-25T01:43:09.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 --2- 172.21.15.78:0/1214269463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a81074e0 0x7f20a81098d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 --2- 172.21.15.78:0/1214269463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a8104bb0 0x7f20a8106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 --2- 172.21.15.78:0/1214269463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a8102280 0x7f20a8104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 -- 172.21.15.78:0/1214269463 >> 172.21.15.78:0/1214269463 conn(0x7f20a80fbbf0 msgr2=0x7f20a80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:09.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 -- 172.21.15.78:0/1214269463 shutdown_connections 2024-04-25T01:43:09.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 -- 172.21.15.78:0/1214269463 wait complete. 2024-04-25T01:43:09.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 Processor -- start 2024-04-25T01:43:09.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 -- start start 2024-04-25T01:43:09.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a8102280 0x7f20a8101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:09.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a8104bb0 0x7f20a8100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:09.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a81074e0 0x7f20a8100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:09.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20a810d450 con 0x7f20a8102280 2024-04-25T01:43:09.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f20a810d2d0 con 0x7f20a8104bb0 2024-04-25T01:43:09.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20b05c1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20a810d5d0 con 0x7f20a81074e0 2024-04-25T01:43:09.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.190+0000 7f20ae35d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a8102280 0x7f20a8101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:09.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f20ae35d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a8102280 0x7f20a8101bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41920/0 (socket says 172.21.15.78:41920) 2024-04-25T01:43:09.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f20ae35d640 1 -- 172.21.15.78:0/1191615079 learned_addr learned my addr 172.21.15.78:0/1191615079 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:09.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f20aeb5e640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a81074e0 0x7f20a8100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:09.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f20adb5c640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a8104bb0 0x7f20a8100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:09.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f20ae35d640 1 -- 172.21.15.78:0/1191615079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a81074e0 msgr2=0x7f20a8100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f20ae35d640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a81074e0 0x7f20a8100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f20ae35d640 1 -- 172.21.15.78:0/1191615079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a8104bb0 msgr2=0x7f20a8100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f20ae35d640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a8104bb0 0x7f20a8100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f20ae35d640 1 -- 172.21.15.78:0/1191615079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a8101000 con 0x7f20a8102280 2024-04-25T01:43:09.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f20adb5c640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a8104bb0 0x7f20a8100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:09.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f20ae35d640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a8102280 0x7f20a8101bf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f20a400d950 tx=0x7f20a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:09.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f209f7fe640 1 -- 172.21.15.78:0/1191615079 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4017070 con 0x7f20a8102280 2024-04-25T01:43:09.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f209f7fe640 1 -- 172.21.15.78:0/1191615079 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a4009c30 con 0x7f20a8102280 2024-04-25T01:43:09.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f20b05c1640 1 -- 172.21.15.78:0/1191615079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a81a1b70 con 0x7f20a8102280 2024-04-25T01:43:09.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f209f7fe640 1 -- 172.21.15.78:0/1191615079 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a400f6b0 con 0x7f20a8102280 2024-04-25T01:43:09.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.191+0000 7f20b05c1640 1 -- 172.21.15.78:0/1191615079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a81a2110 con 0x7f20a8102280 2024-04-25T01:43:09.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.192+0000 7f20b05c1640 1 -- 172.21.15.78:0/1191615079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a80fd6e0 con 0x7f20a8102280 2024-04-25T01:43:09.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.193+0000 7f209f7fe640 1 -- 172.21.15.78:0/1191615079 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f20a40109d0 con 0x7f20a8102280 2024-04-25T01:43:09.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.194+0000 7f209f7fe640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f20940a2c10 0x7f20940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:09.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.194+0000 7f20adb5c640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f20940a2c10 0x7f20940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:09.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.194+0000 7f20adb5c640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f20940a2c10 0x7f20940a50d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20980058c0 tx=0x7f2098005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:09.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.195+0000 7f209f7fe640 1 -- 172.21.15.78:0/1191615079 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f20a40cc840 con 0x7f20a8102280 2024-04-25T01:43:09.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.200+0000 7f209f7fe640 1 -- 172.21.15.78:0/1191615079 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f20a4009de0 con 0x7f20a8102280 2024-04-25T01:43:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.413+0000 7f20b05c1640 1 -- 172.21.15.78:0/1191615079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20a802d170 con 0x7f20a8102280 2024-04-25T01:43:09.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.413+0000 7f209f7fe640 1 -- 172.21.15.78:0/1191615079 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7f20a409bb90 con 0x7f20a8102280 2024-04-25T01:43:09.403 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:09.403 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/1337519666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1337519666},{"type":"v1","addr":"172.21.15.78:6839","nonce":1337519666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T01:43:09.403 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 11 2024-04-25T01:43:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.415+0000 7f20b05c1640 1 -- 172.21.15.78:0/1191615079 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f20940a2c10 msgr2=0x7f20940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.415+0000 7f20b05c1640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f20940a2c10 0x7f20940a50d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20980058c0 tx=0x7f2098005850 comp rx=0 tx=0).stop 2024-04-25T01:43:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.415+0000 7f20b05c1640 1 -- 172.21.15.78:0/1191615079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a8102280 msgr2=0x7f20a8101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.415+0000 7f20b05c1640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a8102280 0x7f20a8101bf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f20a400d950 tx=0x7f20a400de10 comp rx=0 tx=0).stop 2024-04-25T01:43:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.415+0000 7f20b05c1640 1 -- 172.21.15.78:0/1191615079 shutdown_connections 2024-04-25T01:43:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.415+0000 7f20b05c1640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a81074e0 0x7f20a8100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.415+0000 7f20b05c1640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20a8104bb0 0x7f20a8100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.415+0000 7f20b05c1640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f20940a2c10 0x7f20940a50d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.415+0000 7f20b05c1640 1 --2- 172.21.15.78:0/1191615079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a8102280 0x7f20a8101bf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.415+0000 7f20b05c1640 1 -- 172.21.15.78:0/1191615079 >> 172.21.15.78:0/1191615079 conn(0x7f20a80fbbf0 msgr2=0x7f20a8102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.415+0000 7f20b05c1640 1 -- 172.21.15.78:0/1191615079 shutdown_connections 2024-04-25T01:43:09.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:09.415+0000 7f20b05c1640 1 -- 172.21.15.78:0/1191615079 wait complete. 2024-04-25T01:43:09.414 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:43:09 smithi078 ceph-mds[68471]: -- 172.21.15.78:0/1477746353 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56407d081a40 con 0x56407d055000 2024-04-25T01:43:09.414 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:43:09 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-a[68467]: starting mds.a at 2024-04-25T01:43:09.415 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:43:09 smithi078 bash[68452]: b1e800d9d8d9faa251a79571bd4c857502958ffc28cc7f79ae3a123a873dadd9 2024-04-25T01:43:09.415 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:43:09 smithi078 systemd[1]: Started Ceph mds.a for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:43:10.090 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:09 smithi152 ceph-mon[32465]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:10.091 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:10.091 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:10.091 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:10.091 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:43:10.091 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:43:10.091 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:10.091 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:09 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1191615079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:10.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:09 smithi078 ceph-mon[29469]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:10.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:10.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:10.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:10.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:43:10.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:43:10.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:10.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:09 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1191615079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:10.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:09 smithi196 ceph-mon[32978]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:10.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:10.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:10.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:10.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:43:10.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:43:10.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:10.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:09 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1191615079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:11.235 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:10 smithi152 ceph-mon[32465]: Deploying daemon mds.b on smithi152 2024-04-25T01:43:11.235 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:10 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.78:6840/1963410683,v1:172.21.15.78:6841/1963410683] up:boot 2024-04-25T01:43:11.235 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:10 smithi152 ceph-mon[32465]: fsmap 11 up:standby 2024-04-25T01:43:11.235 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:10 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T01:43:11.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:10 smithi078 ceph-mon[29469]: Deploying daemon mds.b on smithi152 2024-04-25T01:43:11.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:10 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.78:6840/1963410683,v1:172.21.15.78:6841/1963410683] up:boot 2024-04-25T01:43:11.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:10 smithi078 ceph-mon[29469]: fsmap 11 up:standby 2024-04-25T01:43:11.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:10 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T01:43:11.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:10 smithi196 ceph-mon[32978]: Deploying daemon mds.b on smithi152 2024-04-25T01:43:11.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:10 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.78:6840/1963410683,v1:172.21.15.78:6841/1963410683] up:boot 2024-04-25T01:43:11.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:10 smithi196 ceph-mon[32978]: fsmap 11 up:standby 2024-04-25T01:43:11.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:10 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T01:43:11.417 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:43:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.540+0000 7efdd097d640 1 -- 172.21.15.78:0/4027412158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdcc102250 msgr2=0x7efdcc108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:11.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.540+0000 7efdd097d640 1 --2- 172.21.15.78:0/4027412158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdcc102250 0x7efdcc108ae0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009ca0 tx=0x7efdc402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.540+0000 7efdd097d640 1 -- 172.21.15.78:0/4027412158 shutdown_connections 2024-04-25T01:43:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.540+0000 7efdd097d640 1 --2- 172.21.15.78:0/4027412158 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdcc102250 0x7efdcc108ae0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.540+0000 7efdd097d640 1 --2- 172.21.15.78:0/4027412158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdcc101890 0x7efdcc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.540+0000 7efdd097d640 1 --2- 172.21.15.78:0/4027412158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdcc1006a0 0x7efdcc100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.540+0000 7efdd097d640 1 -- 172.21.15.78:0/4027412158 >> 172.21.15.78:0/4027412158 conn(0x7efdcc0fbc30 msgr2=0x7efdcc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.540+0000 7efdd097d640 1 -- 172.21.15.78:0/4027412158 shutdown_connections 2024-04-25T01:43:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.540+0000 7efdd097d640 1 -- 172.21.15.78:0/4027412158 wait complete. 2024-04-25T01:43:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.541+0000 7efdd097d640 1 Processor -- start 2024-04-25T01:43:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.541+0000 7efdd097d640 1 -- start start 2024-04-25T01:43:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.541+0000 7efdd097d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdcc1006a0 0x7efdcc1968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.541+0000 7efdd097d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdcc101890 0x7efdcc196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.541+0000 7efdd097d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdcc102250 0x7efdcc19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.541+0000 7efdd097d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc10c350 con 0x7efdcc1006a0 2024-04-25T01:43:11.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.541+0000 7efdd097d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc10c1d0 con 0x7efdcc101890 2024-04-25T01:43:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.541+0000 7efdd097d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc10c4d0 con 0x7efdcc102250 2024-04-25T01:43:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.541+0000 7efdca59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdcc1006a0 0x7efdcc1968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.541+0000 7efdca59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdcc1006a0 0x7efdcc1968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56862/0 (socket says 172.21.15.78:56862) 2024-04-25T01:43:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.541+0000 7efdca59c640 1 -- 172.21.15.78:0/263038605 learned_addr learned my addr 172.21.15.78:0/263038605 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.541+0000 7efdc3fff640 1 --2- 172.21.15.78:0/263038605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdcc101890 0x7efdcc196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.541+0000 7efdcad9d640 1 --2- 172.21.15.78:0/263038605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdcc102250 0x7efdcc19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.542+0000 7efdca59c640 1 -- 172.21.15.78:0/263038605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdcc102250 msgr2=0x7efdcc19de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.542+0000 7efdca59c640 1 --2- 172.21.15.78:0/263038605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdcc102250 0x7efdcc19de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.542+0000 7efdca59c640 1 -- 172.21.15.78:0/263038605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdcc101890 msgr2=0x7efdcc196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.542+0000 7efdca59c640 1 --2- 172.21.15.78:0/263038605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdcc101890 0x7efdcc196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.542+0000 7efdca59c640 1 -- 172.21.15.78:0/263038605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdcc19e4c0 con 0x7efdcc1006a0 2024-04-25T01:43:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.542+0000 7efdca59c640 1 --2- 172.21.15.78:0/263038605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdcc1006a0 0x7efdcc1968a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00c970 tx=0x7efdbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:11.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.542+0000 7efdc37fe640 1 -- 172.21.15.78:0/263038605 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc007c10 con 0x7efdcc1006a0 2024-04-25T01:43:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.542+0000 7efdc37fe640 1 -- 172.21.15.78:0/263038605 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdbc007db0 con 0x7efdcc1006a0 2024-04-25T01:43:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.542+0000 7efdc37fe640 1 -- 172.21.15.78:0/263038605 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc00f940 con 0x7efdcc1006a0 2024-04-25T01:43:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.542+0000 7efdd097d640 1 -- 172.21.15.78:0/263038605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdcc19e7b0 con 0x7efdcc1006a0 2024-04-25T01:43:11.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.542+0000 7efdd097d640 1 -- 172.21.15.78:0/263038605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdcc19ed00 con 0x7efdcc1006a0 2024-04-25T01:43:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.543+0000 7efdd097d640 1 -- 172.21.15.78:0/263038605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdcc1078f0 con 0x7efdcc1006a0 2024-04-25T01:43:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.544+0000 7efdc37fe640 1 -- 172.21.15.78:0/263038605 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efdbc00fae0 con 0x7efdcc1006a0 2024-04-25T01:43:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.545+0000 7efdc37fe640 1 --2- 172.21.15.78:0/263038605 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efdb80a2b90 0x7efdb80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.545+0000 7efdc3fff640 1 --2- 172.21.15.78:0/263038605 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efdb80a2b90 0x7efdb80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.545+0000 7efdc37fe640 1 -- 172.21.15.78:0/263038605 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efdbc0cef70 con 0x7efdcc1006a0 2024-04-25T01:43:11.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.545+0000 7efdc3fff640 1 --2- 172.21.15.78:0/263038605 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efdb80a2b90 0x7efdb80a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efdcc197db0 tx=0x7efdb4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.549+0000 7efdc37fe640 1 -- 172.21.15.78:0/263038605 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efdbc09c400 con 0x7efdcc1006a0 2024-04-25T01:43:11.546 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:43:11 smithi152 systemd[1]: Starting Ceph mds.b for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:43:11.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.760+0000 7efdd097d640 1 -- 172.21.15.78:0/263038605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdcc02d170 con 0x7efdcc1006a0 2024-04-25T01:43:11.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.761+0000 7efdc37fe640 1 -- 172.21.15.78:0/263038605 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7efdbc09bd00 con 0x7efdcc1006a0 2024-04-25T01:43:11.750 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:11.750 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/1337519666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1337519666},{"type":"v1","addr":"172.21.15.78:6839","nonce":1337519666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1963410683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1963410683},{"type":"v1","addr":"172.21.15.78:6841","nonce":1963410683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T01:43:11.751 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 12 2024-04-25T01:43:11.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.763+0000 7efdd097d640 1 -- 172.21.15.78:0/263038605 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efdb80a2b90 msgr2=0x7efdb80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.763+0000 7efdd097d640 1 --2- 172.21.15.78:0/263038605 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efdb80a2b90 0x7efdb80a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efdcc197db0 tx=0x7efdb4005f90 comp rx=0 tx=0).stop 2024-04-25T01:43:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.763+0000 7efdd097d640 1 -- 172.21.15.78:0/263038605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdcc1006a0 msgr2=0x7efdcc1968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.763+0000 7efdd097d640 1 --2- 172.21.15.78:0/263038605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdcc1006a0 0x7efdcc1968a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00c970 tx=0x7efdbc00ce30 comp rx=0 tx=0).stop 2024-04-25T01:43:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.763+0000 7efdd097d640 1 -- 172.21.15.78:0/263038605 shutdown_connections 2024-04-25T01:43:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.763+0000 7efdd097d640 1 --2- 172.21.15.78:0/263038605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efdcc102250 0x7efdcc19de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.763+0000 7efdd097d640 1 --2- 172.21.15.78:0/263038605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdcc101890 0x7efdcc196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.763+0000 7efdd097d640 1 --2- 172.21.15.78:0/263038605 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efdb80a2b90 0x7efdb80a5050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.763+0000 7efdd097d640 1 --2- 172.21.15.78:0/263038605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efdcc1006a0 0x7efdcc1968a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.763+0000 7efdd097d640 1 -- 172.21.15.78:0/263038605 >> 172.21.15.78:0/263038605 conn(0x7efdcc0fbc30 msgr2=0x7efdcc104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.763+0000 7efdd097d640 1 -- 172.21.15.78:0/263038605 shutdown_connections 2024-04-25T01:43:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:11.763+0000 7efdd097d640 1 -- 172.21.15.78:0/263038605 wait complete. 2024-04-25T01:43:11.894 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:43:11 smithi152 podman[62789]: 2024-04-25 01:43:11.545305254 +0000 UTC m=+0.018944468 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:43:11.894 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:43:11 smithi152 podman[62789]: 2024-04-25 01:43:11.660148041 +0000 UTC m=+0.133787249 container create 87728433819dba640e7aa2b4a417f3e4d35901504b41035a5489e72442da21f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-25T01:43:11.894 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:43:11 smithi152 podman[62789]: 2024-04-25 01:43:11.90474459 +0000 UTC m=+0.378383798 container init 87728433819dba640e7aa2b4a417f3e4d35901504b41035a5489e72442da21f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-b, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T01:43:12.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:11 smithi078 ceph-mon[29469]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:12.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:11 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/263038605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:12.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:11 smithi152 ceph-mon[32465]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:12.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:11 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/263038605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:12.297 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:43:11 smithi152 podman[62789]: 2024-04-25 01:43:11.907655443 +0000 UTC m=+0.381294656 container start 87728433819dba640e7aa2b4a417f3e4d35901504b41035a5489e72442da21f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-25T01:43:12.297 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:43:11 smithi152 ceph-mds[62808]: -- 172.21.15.152:0/1500208130 <== mon.2 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644198b5a40 con 0x564419888c00 2024-04-25T01:43:12.297 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:43:11 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-b[62804]: starting mds.b at 2024-04-25T01:43:12.297 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:43:12 smithi152 bash[62789]: 87728433819dba640e7aa2b4a417f3e4d35901504b41035a5489e72442da21f7 2024-04-25T01:43:12.297 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:43:12 smithi152 systemd[1]: Started Ceph mds.b for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:43:12.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:11 smithi196 ceph-mon[32978]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:12.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:11 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/263038605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:12 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] up:boot 2024-04-25T01:43:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:12 smithi078 ceph-mon[29469]: fsmap 12 up:standby 2024-04-25T01:43:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:12 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T01:43:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:12 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:12 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:12 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:12 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:12 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:12 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:43:13.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:12 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] up:boot 2024-04-25T01:43:13.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:12 smithi152 ceph-mon[32465]: fsmap 12 up:standby 2024-04-25T01:43:13.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:12 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T01:43:13.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:12 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:12 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:12 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:12 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:12 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:12 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:43:13.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:12 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] up:boot 2024-04-25T01:43:13.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:12 smithi196 ceph-mon[32978]: fsmap 12 up:standby 2024-04-25T01:43:13.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:12 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T01:43:13.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:12 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:12 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:12 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:12 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:12 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:13.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:12 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:43:13.764 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:43:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.883+0000 7f78bc81b640 1 -- 172.21.15.78:0/2912716105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b8101870 msgr2=0x7f78b8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.883+0000 7f78bc81b640 1 --2- 172.21.15.78:0/2912716105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b8101870 0x7f78b8101cf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009900 tx=0x7f78ac02a410 comp rx=0 tx=0).stop 2024-04-25T01:43:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.883+0000 7f78bc81b640 1 -- 172.21.15.78:0/2912716105 shutdown_connections 2024-04-25T01:43:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.883+0000 7f78bc81b640 1 --2- 172.21.15.78:0/2912716105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b8102230 0x7f78b8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.883+0000 7f78bc81b640 1 --2- 172.21.15.78:0/2912716105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b8101870 0x7f78b8101cf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.883+0000 7f78bc81b640 1 --2- 172.21.15.78:0/2912716105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b8100680 0x7f78b8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.883+0000 7f78bc81b640 1 -- 172.21.15.78:0/2912716105 >> 172.21.15.78:0/2912716105 conn(0x7f78b80fbbf0 msgr2=0x7f78b80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.883+0000 7f78bc81b640 1 -- 172.21.15.78:0/2912716105 shutdown_connections 2024-04-25T01:43:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.883+0000 7f78bc81b640 1 -- 172.21.15.78:0/2912716105 wait complete. 2024-04-25T01:43:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78bc81b640 1 Processor -- start 2024-04-25T01:43:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78bc81b640 1 -- start start 2024-04-25T01:43:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78bc81b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b8100680 0x7f78b80694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78bc81b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b8101870 0x7f78b819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78bc81b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b8102230 0x7f78b819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78bc81b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78b810c350 con 0x7f78b8102230 2024-04-25T01:43:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78bc81b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f78b810c1d0 con 0x7f78b8101870 2024-04-25T01:43:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78bc81b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78b810c4d0 con 0x7f78b8100680 2024-04-25T01:43:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78b6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b8102230 0x7f78b819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78b6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b8102230 0x7f78b819ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56880/0 (socket says 172.21.15.78:56880) 2024-04-25T01:43:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78b6d9d640 1 -- 172.21.15.78:0/1437801923 learned_addr learned my addr 172.21.15.78:0/1437801923 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78b659c640 1 --2- 172.21.15.78:0/1437801923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b8100680 0x7f78b80694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78b5d9b640 1 --2- 172.21.15.78:0/1437801923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b8101870 0x7f78b819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78b6d9d640 1 -- 172.21.15.78:0/1437801923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b8100680 msgr2=0x7f78b80694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78b6d9d640 1 --2- 172.21.15.78:0/1437801923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b8100680 0x7f78b80694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78b6d9d640 1 -- 172.21.15.78:0/1437801923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b8101870 msgr2=0x7f78b819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78b6d9d640 1 --2- 172.21.15.78:0/1437801923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b8101870 0x7f78b819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78b6d9d640 1 -- 172.21.15.78:0/1437801923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b819f6e0 con 0x7f78b8102230 2024-04-25T01:43:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.884+0000 7f78b6d9d640 1 --2- 172.21.15.78:0/1437801923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b8102230 0x7f78b819ee20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009fd0 tx=0x7f78b000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:13.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.885+0000 7f78a77fe640 1 -- 172.21.15.78:0/1437801923 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0017070 con 0x7f78b8102230 2024-04-25T01:43:13.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.885+0000 7f78a77fe640 1 -- 172.21.15.78:0/1437801923 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b0003e50 con 0x7f78b8102230 2024-04-25T01:43:13.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.885+0000 7f78bc81b640 1 -- 172.21.15.78:0/1437801923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b819f9d0 con 0x7f78b8102230 2024-04-25T01:43:13.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.885+0000 7f78a77fe640 1 -- 172.21.15.78:0/1437801923 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b00106c0 con 0x7f78b8102230 2024-04-25T01:43:13.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.885+0000 7f78bc81b640 1 -- 172.21.15.78:0/1437801923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b819fef0 con 0x7f78b8102230 2024-04-25T01:43:13.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.886+0000 7f78bc81b640 1 -- 172.21.15.78:0/1437801923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b81079a0 con 0x7f78b8102230 2024-04-25T01:43:13.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.887+0000 7f78a77fe640 1 -- 172.21.15.78:0/1437801923 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f78b0003970 con 0x7f78b8102230 2024-04-25T01:43:13.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.887+0000 7f78a77fe640 1 --2- 172.21.15.78:0/1437801923 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f78a00a2c10 0x7f78a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:13.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.887+0000 7f78a77fe640 1 -- 172.21.15.78:0/1437801923 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f78b00ce800 con 0x7f78b8102230 2024-04-25T01:43:13.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.887+0000 7f78b659c640 1 --2- 172.21.15.78:0/1437801923 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f78a00a2c10 0x7f78a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:13.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.888+0000 7f78b659c640 1 --2- 172.21.15.78:0/1437801923 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f78a00a2c10 0x7f78a00a50d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f78b81016d0 tx=0x7f78a8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:13.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:13.893+0000 7f78a77fe640 1 -- 172.21.15.78:0/1437801923 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f78b009b5b0 con 0x7f78b8102230 2024-04-25T01:43:14.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.123+0000 7f78bc81b640 1 -- 172.21.15.78:0/1437801923 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78b802d170 con 0x7f78b8102230 2024-04-25T01:43:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.124+0000 7f78a77fe640 1 -- 172.21.15.78:0/1437801923 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10491 (secure 0 0 0) 0x7f78b009f8b0 con 0x7f78b8102230 2024-04-25T01:43:14.114 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:14.114 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/1337519666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1337519666},{"type":"v1","addr":"172.21.15.78:6839","nonce":1337519666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1963410683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1963410683},{"type":"v1","addr":"172.21.15.78:6841","nonce":1963410683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/1969344843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1969344843},{"type":"v1","addr":"172.21.15.152:6839","nonce":1969344843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T01:43:14.114 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 13 2024-04-25T01:43:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.126+0000 7f78bc81b640 1 -- 172.21.15.78:0/1437801923 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f78a00a2c10 msgr2=0x7f78a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.126+0000 7f78bc81b640 1 --2- 172.21.15.78:0/1437801923 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f78a00a2c10 0x7f78a00a50d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f78b81016d0 tx=0x7f78a8005f90 comp rx=0 tx=0).stop 2024-04-25T01:43:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.126+0000 7f78bc81b640 1 -- 172.21.15.78:0/1437801923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b8102230 msgr2=0x7f78b819ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.126+0000 7f78bc81b640 1 --2- 172.21.15.78:0/1437801923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b8102230 0x7f78b819ee20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009fd0 tx=0x7f78b000b260 comp rx=0 tx=0).stop 2024-04-25T01:43:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.126+0000 7f78bc81b640 1 -- 172.21.15.78:0/1437801923 shutdown_connections 2024-04-25T01:43:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.126+0000 7f78bc81b640 1 --2- 172.21.15.78:0/1437801923 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f78a00a2c10 0x7f78a00a50d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.126+0000 7f78bc81b640 1 --2- 172.21.15.78:0/1437801923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b8102230 0x7f78b819ee20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.126+0000 7f78bc81b640 1 --2- 172.21.15.78:0/1437801923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b8101870 0x7f78b819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.126+0000 7f78bc81b640 1 --2- 172.21.15.78:0/1437801923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b8100680 0x7f78b80694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.126+0000 7f78bc81b640 1 -- 172.21.15.78:0/1437801923 >> 172.21.15.78:0/1437801923 conn(0x7f78b80fbbf0 msgr2=0x7f78b8104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:14.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.126+0000 7f78bc81b640 1 -- 172.21.15.78:0/1437801923 shutdown_connections 2024-04-25T01:43:14.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.126+0000 7f78bc81b640 1 -- 172.21.15.78:0/1437801923 wait complete. 2024-04-25T01:43:14.126 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:13 smithi078 ceph-mon[29469]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:14.126 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:13 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:14.126 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:13 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:14.128 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:43:14.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.235+0000 7f82580d2640 1 -- 172.21.15.78:0/2945566734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82501006a0 msgr2=0x7f8250100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.235+0000 7f82580d2640 1 --2- 172.21.15.78:0/2945566734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82501006a0 0x7f8250100ac0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8248009ca0 tx=0x7f824802a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.235+0000 7f82580d2640 1 -- 172.21.15.78:0/2945566734 shutdown_connections 2024-04-25T01:43:14.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.235+0000 7f82580d2640 1 --2- 172.21.15.78:0/2945566734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8250102250 0x7f8250108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.235+0000 7f82580d2640 1 --2- 172.21.15.78:0/2945566734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8250101890 0x7f8250101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.235+0000 7f82580d2640 1 --2- 172.21.15.78:0/2945566734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82501006a0 0x7f8250100ac0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.235+0000 7f82580d2640 1 -- 172.21.15.78:0/2945566734 >> 172.21.15.78:0/2945566734 conn(0x7f82500fbc30 msgr2=0x7f82500fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:14.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.235+0000 7f82580d2640 1 -- 172.21.15.78:0/2945566734 shutdown_connections 2024-04-25T01:43:14.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.235+0000 7f82580d2640 1 -- 172.21.15.78:0/2945566734 wait complete. 2024-04-25T01:43:14.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.235+0000 7f82580d2640 1 Processor -- start 2024-04-25T01:43:14.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.235+0000 7f82580d2640 1 -- start start 2024-04-25T01:43:14.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f82580d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82501006a0 0x7f825006a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:14.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f82580d2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8250101890 0x7f825019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:14.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f82580d2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8250102250 0x7f825019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:14.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f82580d2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f825010c320 con 0x7f82501006a0 2024-04-25T01:43:14.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f82580d2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f825010c1a0 con 0x7f8250101890 2024-04-25T01:43:14.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f82580d2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f825010c4a0 con 0x7f8250102250 2024-04-25T01:43:14.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f8255e6e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82501006a0 0x7f825006a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:14.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f8255e6e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82501006a0 0x7f825006a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56898/0 (socket says 172.21.15.78:56898) 2024-04-25T01:43:14.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f8255e6e640 1 -- 172.21.15.78:0/2206806990 learned_addr learned my addr 172.21.15.78:0/2206806990 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:14.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f825666f640 1 --2- 172.21.15.78:0/2206806990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8250102250 0x7f825019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:14.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f8255e6e640 1 -- 172.21.15.78:0/2206806990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8250102250 msgr2=0x7f825019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f8255e6e640 1 --2- 172.21.15.78:0/2206806990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8250102250 0x7f825019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f8255e6e640 1 -- 172.21.15.78:0/2206806990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8250101890 msgr2=0x7f825019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f8255e6e640 1 --2- 172.21.15.78:0/2206806990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8250101890 0x7f825019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f8255e6e640 1 -- 172.21.15.78:0/2206806990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825019f6e0 con 0x7f82501006a0 2024-04-25T01:43:14.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f8255e6e640 1 --2- 172.21.15.78:0/2206806990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82501006a0 0x7f825006a950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8248009bf0 tx=0x7f8248031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:14.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f8246ffd640 1 -- 172.21.15.78:0/2206806990 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824802fae0 con 0x7f82501006a0 2024-04-25T01:43:14.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f8246ffd640 1 -- 172.21.15.78:0/2206806990 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f824802fc80 con 0x7f82501006a0 2024-04-25T01:43:14.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f82580d2640 1 -- 172.21.15.78:0/2206806990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825019f910 con 0x7f82501006a0 2024-04-25T01:43:14.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f8246ffd640 1 -- 172.21.15.78:0/2206806990 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82480467e0 con 0x7f82501006a0 2024-04-25T01:43:14.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.236+0000 7f82580d2640 1 -- 172.21.15.78:0/2206806990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825019fe60 con 0x7f82501006a0 2024-04-25T01:43:14.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.237+0000 7f82580d2640 1 -- 172.21.15.78:0/2206806990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8250107960 con 0x7f82501006a0 2024-04-25T01:43:14.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.238+0000 7f8246ffd640 1 -- 172.21.15.78:0/2206806990 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8248030b30 con 0x7f82501006a0 2024-04-25T01:43:14.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.239+0000 7f8246ffd640 1 --2- 172.21.15.78:0/2206806990 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f823c0a2b90 0x7f823c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:14.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.239+0000 7f825566d640 1 --2- 172.21.15.78:0/2206806990 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f823c0a2b90 0x7f823c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:14.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.239+0000 7f8246ffd640 1 -- 172.21.15.78:0/2206806990 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f82480f0090 con 0x7f82501006a0 2024-04-25T01:43:14.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.239+0000 7f825566d640 1 --2- 172.21.15.78:0/2206806990 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f823c0a2b90 0x7f823c0a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82400058c0 tx=0x7f8240005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:14.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.243+0000 7f8246ffd640 1 -- 172.21.15.78:0/2206806990 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f82480bd490 con 0x7f82501006a0 2024-04-25T01:43:14.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:13 smithi152 ceph-mon[32465]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:14.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:13 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:14.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:13 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:14.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:13 smithi196 ceph-mon[32978]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:14.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:13 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:14.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:13 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:14.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.424+0000 7f82580d2640 1 -- 172.21.15.78:0/2206806990 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f825002d3c0 con 0x7f82501006a0 2024-04-25T01:43:14.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.425+0000 7f8246ffd640 1 -- 172.21.15.78:0/2206806990 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18479 (secure 0 0 0) 0x7f82480bcd90 con 0x7f82501006a0 2024-04-25T01:43:14.416 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:14.417 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":82,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:42:28.429191+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:43:14.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.427+0000 7f82580d2640 1 -- 172.21.15.78:0/2206806990 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f823c0a2b90 msgr2=0x7f823c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.427+0000 7f82580d2640 1 --2- 172.21.15.78:0/2206806990 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f823c0a2b90 0x7f823c0a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82400058c0 tx=0x7f8240005850 comp rx=0 tx=0).stop 2024-04-25T01:43:14.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.428+0000 7f82580d2640 1 -- 172.21.15.78:0/2206806990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82501006a0 msgr2=0x7f825006a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.428+0000 7f82580d2640 1 --2- 172.21.15.78:0/2206806990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82501006a0 0x7f825006a950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8248009bf0 tx=0x7f8248031ee0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.428+0000 7f82580d2640 1 -- 172.21.15.78:0/2206806990 shutdown_connections 2024-04-25T01:43:14.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.428+0000 7f82580d2640 1 --2- 172.21.15.78:0/2206806990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8250102250 0x7f825019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.428+0000 7f82580d2640 1 --2- 172.21.15.78:0/2206806990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8250101890 0x7f825019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.428+0000 7f82580d2640 1 --2- 172.21.15.78:0/2206806990 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f823c0a2b90 0x7f823c0a5050 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.428+0000 7f82580d2640 1 --2- 172.21.15.78:0/2206806990 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82501006a0 0x7f825006a950 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.428+0000 7f82580d2640 1 -- 172.21.15.78:0/2206806990 >> 172.21.15.78:0/2206806990 conn(0x7f82500fbc30 msgr2=0x7f8250104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:14.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.428+0000 7f82580d2640 1 -- 172.21.15.78:0/2206806990 shutdown_connections 2024-04-25T01:43:14.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.428+0000 7f82580d2640 1 -- 172.21.15.78:0/2206806990 wait complete. 2024-04-25T01:43:14.431 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:38:39.136418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:43:14.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:43:14.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.542+0000 7f693926b640 1 -- 172.21.15.78:0/1769366052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69341012d0 msgr2=0x7f6934108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.542+0000 7f693926b640 1 --2- 172.21.15.78:0/1769366052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69341012d0 0x7f6934108b70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f692c009cd0 tx=0x7f692c02a8e0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.542+0000 7f693926b640 1 -- 172.21.15.78:0/1769366052 shutdown_connections 2024-04-25T01:43:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.542+0000 7f693926b640 1 --2- 172.21.15.78:0/1769366052 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69341012d0 0x7f6934108b70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.542+0000 7f693926b640 1 --2- 172.21.15.78:0/1769366052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6934100910 0x7f6934100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.542+0000 7f693926b640 1 --2- 172.21.15.78:0/1769366052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6934101c70 0x7f6934102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.542+0000 7f693926b640 1 -- 172.21.15.78:0/1769366052 >> 172.21.15.78:0/1769366052 conn(0x7f69340fb380 msgr2=0x7f69340fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:14.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.542+0000 7f693926b640 1 -- 172.21.15.78:0/1769366052 shutdown_connections 2024-04-25T01:43:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.542+0000 7f693926b640 1 -- 172.21.15.78:0/1769366052 wait complete. 2024-04-25T01:43:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.542+0000 7f693926b640 1 Processor -- start 2024-04-25T01:43:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f693926b640 1 -- start start 2024-04-25T01:43:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f693926b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6934100910 0x7f6934197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f693926b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69341012d0 0x7f6934197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f693926b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6934101c70 0x7f693419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f693926b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f693410c320 con 0x7f6934100910 2024-04-25T01:43:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f693926b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f693410c1a0 con 0x7f6934101c70 2024-04-25T01:43:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f693926b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f693410c4a0 con 0x7f69341012d0 2024-04-25T01:43:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f6932ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6934100910 0x7f6934197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f6932ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6934100910 0x7f6934197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56906/0 (socket says 172.21.15.78:56906) 2024-04-25T01:43:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f6932ffd640 1 -- 172.21.15.78:0/3249522139 learned_addr learned my addr 172.21.15.78:0/3249522139 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f69327fc640 1 --2- 172.21.15.78:0/3249522139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69341012d0 0x7f6934197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f6932ffd640 1 -- 172.21.15.78:0/3249522139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69341012d0 msgr2=0x7f6934197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f6932ffd640 1 --2- 172.21.15.78:0/3249522139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69341012d0 0x7f6934197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f6932ffd640 1 -- 172.21.15.78:0/3249522139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6934101c70 msgr2=0x7f693419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f6932ffd640 1 --2- 172.21.15.78:0/3249522139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6934101c70 0x7f693419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f6932ffd640 1 -- 172.21.15.78:0/3249522139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693419f6b0 con 0x7f6934100910 2024-04-25T01:43:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f6932ffd640 1 --2- 172.21.15.78:0/3249522139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6934100910 0x7f6934197a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f692400c970 tx=0x7f692400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f691bfff640 1 -- 172.21.15.78:0/3249522139 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924004220 con 0x7f6934100910 2024-04-25T01:43:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f693926b640 1 -- 172.21.15.78:0/3249522139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693419f9a0 con 0x7f6934100910 2024-04-25T01:43:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f691bfff640 1 -- 172.21.15.78:0/3249522139 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69240043c0 con 0x7f6934100910 2024-04-25T01:43:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f691bfff640 1 -- 172.21.15.78:0/3249522139 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924004a00 con 0x7f6934100910 2024-04-25T01:43:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.543+0000 7f693926b640 1 -- 172.21.15.78:0/3249522139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693419fef0 con 0x7f6934100910 2024-04-25T01:43:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.544+0000 7f693926b640 1 -- 172.21.15.78:0/3249522139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69341a0180 con 0x7f6934100910 2024-04-25T01:43:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.545+0000 7f691bfff640 1 -- 172.21.15.78:0/3249522139 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f69240079b0 con 0x7f6934100910 2024-04-25T01:43:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.546+0000 7f691bfff640 1 --2- 172.21.15.78:0/3249522139 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f691c0a2b90 0x7f691c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:14.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.546+0000 7f69327fc640 1 --2- 172.21.15.78:0/3249522139 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f691c0a2b90 0x7f691c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.546+0000 7f691bfff640 1 -- 172.21.15.78:0/3249522139 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f69240ce9f0 con 0x7f6934100910 2024-04-25T01:43:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.546+0000 7f69327fc640 1 --2- 172.21.15.78:0/3249522139 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f691c0a2b90 0x7f691c0a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6934198fa0 tx=0x7f692800a4c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:14.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.550+0000 7f691bfff640 1 -- 172.21.15.78:0/3249522139 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6924014030 con 0x7f6934100910 2024-04-25T01:43:14.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.733+0000 7f693926b640 1 -- 172.21.15.78:0/3249522139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f69341a0350 con 0x7f6934100910 2024-04-25T01:43:14.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.734+0000 7f691bfff640 1 -- 172.21.15.78:0/3249522139 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f692409bdf0 con 0x7f6934100910 2024-04-25T01:43:14.725 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T01:43:14.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.736+0000 7f693926b640 1 -- 172.21.15.78:0/3249522139 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f691c0a2b90 msgr2=0x7f691c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.736+0000 7f693926b640 1 --2- 172.21.15.78:0/3249522139 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f691c0a2b90 0x7f691c0a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6934198fa0 tx=0x7f692800a4c0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.736+0000 7f693926b640 1 -- 172.21.15.78:0/3249522139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6934100910 msgr2=0x7f6934197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.736+0000 7f693926b640 1 --2- 172.21.15.78:0/3249522139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6934100910 0x7f6934197a90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f692400c970 tx=0x7f692400ce30 comp rx=0 tx=0).stop 2024-04-25T01:43:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.736+0000 7f693926b640 1 -- 172.21.15.78:0/3249522139 shutdown_connections 2024-04-25T01:43:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.736+0000 7f693926b640 1 --2- 172.21.15.78:0/3249522139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6934101c70 0x7f693419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.736+0000 7f693926b640 1 --2- 172.21.15.78:0/3249522139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69341012d0 0x7f6934197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.736+0000 7f693926b640 1 --2- 172.21.15.78:0/3249522139 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f691c0a2b90 0x7f691c0a5050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.736+0000 7f693926b640 1 --2- 172.21.15.78:0/3249522139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6934100910 0x7f6934197a90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.736+0000 7f693926b640 1 -- 172.21.15.78:0/3249522139 >> 172.21.15.78:0/3249522139 conn(0x7f69340fb380 msgr2=0x7f69340fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.736+0000 7f693926b640 1 -- 172.21.15.78:0/3249522139 shutdown_connections 2024-04-25T01:43:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.736+0000 7f693926b640 1 -- 172.21.15.78:0/3249522139 wait complete. 2024-04-25T01:43:14.738 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T01:43:14.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.848+0000 7f6ebe67b640 1 -- 172.21.15.78:0/2725624544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6eb8102230 msgr2=0x7f6eb8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.848+0000 7f6ebe67b640 1 --2- 172.21.15.78:0/2725624544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6eb8102230 0x7f6eb8108ac0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009ca0 tx=0x7f6eb002a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.848+0000 7f6ebe67b640 1 -- 172.21.15.78:0/2725624544 shutdown_connections 2024-04-25T01:43:14.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.848+0000 7f6ebe67b640 1 --2- 172.21.15.78:0/2725624544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6eb8102230 0x7f6eb8108ac0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.848+0000 7f6ebe67b640 1 --2- 172.21.15.78:0/2725624544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb8101870 0x7f6eb8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.848+0000 7f6ebe67b640 1 --2- 172.21.15.78:0/2725624544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eb8100680 0x7f6eb8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.848+0000 7f6ebe67b640 1 -- 172.21.15.78:0/2725624544 >> 172.21.15.78:0/2725624544 conn(0x7f6eb80fbbf0 msgr2=0x7f6eb80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.848+0000 7f6ebe67b640 1 -- 172.21.15.78:0/2725624544 shutdown_connections 2024-04-25T01:43:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.848+0000 7f6ebe67b640 1 -- 172.21.15.78:0/2725624544 wait complete. 2024-04-25T01:43:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.848+0000 7f6ebe67b640 1 Processor -- start 2024-04-25T01:43:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.848+0000 7f6ebe67b640 1 -- start start 2024-04-25T01:43:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebe67b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb8100680 0x7f6eb8197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebe67b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eb8101870 0x7f6eb8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebe67b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6eb8102230 0x7f6eb819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebe67b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb810c2e0 con 0x7f6eb8102230 2024-04-25T01:43:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebe67b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb810c160 con 0x7f6eb8100680 2024-04-25T01:43:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebe67b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb810c460 con 0x7f6eb8101870 2024-04-25T01:43:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebcc18640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6eb8102230 0x7f6eb819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebcc18640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6eb8102230 0x7f6eb819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56920/0 (socket says 172.21.15.78:56920) 2024-04-25T01:43:14.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebcc18640 1 -- 172.21.15.78:0/993368529 learned_addr learned my addr 172.21.15.78:0/993368529 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:14.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6eb7fff640 1 --2- 172.21.15.78:0/993368529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb8100680 0x7f6eb8197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:14.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6eb77fe640 1 --2- 172.21.15.78:0/993368529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eb8101870 0x7f6eb8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:14.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebcc18640 1 -- 172.21.15.78:0/993368529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eb8101870 msgr2=0x7f6eb8197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebcc18640 1 --2- 172.21.15.78:0/993368529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eb8101870 0x7f6eb8197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebcc18640 1 -- 172.21.15.78:0/993368529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb8100680 msgr2=0x7f6eb8197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:14.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebcc18640 1 --2- 172.21.15.78:0/993368529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb8100680 0x7f6eb8197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:14.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebcc18640 1 -- 172.21.15.78:0/993368529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb819f680 con 0x7f6eb8102230 2024-04-25T01:43:14.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebcc18640 1 --2- 172.21.15.78:0/993368529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6eb8102230 0x7f6eb819f010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0005e30 tx=0x7f6eb0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:14.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6eb57fa640 1 -- 172.21.15.78:0/993368529 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0003c30 con 0x7f6eb8102230 2024-04-25T01:43:14.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6eb57fa640 1 -- 172.21.15.78:0/993368529 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6eb0003dd0 con 0x7f6eb8102230 2024-04-25T01:43:14.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebe67b640 1 -- 172.21.15.78:0/993368529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb819f910 con 0x7f6eb8102230 2024-04-25T01:43:14.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6eb57fa640 1 -- 172.21.15.78:0/993368529 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb002fde0 con 0x7f6eb8102230 2024-04-25T01:43:14.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.849+0000 7f6ebe67b640 1 -- 172.21.15.78:0/993368529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb819fe60 con 0x7f6eb8102230 2024-04-25T01:43:14.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.850+0000 7f6ebe67b640 1 -- 172.21.15.78:0/993368529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb81a0030 con 0x7f6eb8102230 2024-04-25T01:43:14.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.852+0000 7f6eb57fa640 1 -- 172.21.15.78:0/993368529 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6eb0004050 con 0x7f6eb8102230 2024-04-25T01:43:14.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.853+0000 7f6eb57fa640 1 --2- 172.21.15.78:0/993368529 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6ea00a2b90 0x7f6ea00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:14.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.853+0000 7f6eb7fff640 1 --2- 172.21.15.78:0/993368529 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6ea00a2b90 0x7f6ea00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:14.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.853+0000 7f6eb57fa640 1 -- 172.21.15.78:0/993368529 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6eb00effd0 con 0x7f6eb8102230 2024-04-25T01:43:14.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.853+0000 7f6eb7fff640 1 --2- 172.21.15.78:0/993368529 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6ea00a2b90 0x7f6ea00a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6eb81016d0 tx=0x7f6ea800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:14.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:14.856+0000 7f6eb57fa640 1 -- 172.21.15.78:0/993368529 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6eb00bd3d0 con 0x7f6eb8102230 2024-04-25T01:43:15.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:14 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1437801923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:15.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:14 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2206806990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:15.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:15.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:43:15.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:14 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:43:15.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:14 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3249522139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:43:15.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.044+0000 7f6ebe67b640 1 -- 172.21.15.78:0/993368529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6eb81a0320 con 0x7f6eb8102230 2024-04-25T01:43:15.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.044+0000 7f6eb57fa640 1 -- 172.21.15.78:0/993368529 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f6eb00bccd0 con 0x7f6eb8102230 2024-04-25T01:43:15.035 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:15.035 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-04-25T01:43:15.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:43:15.035 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-04-25T01:43:15.035 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-04-25T01:43:15.036 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:43:15.036 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-04-25T01:43:15.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.047+0000 7f6ebe67b640 1 -- 172.21.15.78:0/993368529 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6ea00a2b90 msgr2=0x7f6ea00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:15.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.047+0000 7f6ebe67b640 1 --2- 172.21.15.78:0/993368529 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6ea00a2b90 0x7f6ea00a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6eb81016d0 tx=0x7f6ea800b390 comp rx=0 tx=0).stop 2024-04-25T01:43:15.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.047+0000 7f6ebe67b640 1 -- 172.21.15.78:0/993368529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6eb8102230 msgr2=0x7f6eb819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:15.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.047+0000 7f6ebe67b640 1 --2- 172.21.15.78:0/993368529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6eb8102230 0x7f6eb819f010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0005e30 tx=0x7f6eb0031ee0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.047+0000 7f6ebe67b640 1 -- 172.21.15.78:0/993368529 shutdown_connections 2024-04-25T01:43:15.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.047+0000 7f6ebe67b640 1 --2- 172.21.15.78:0/993368529 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6ea00a2b90 0x7f6ea00a5050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.047+0000 7f6ebe67b640 1 --2- 172.21.15.78:0/993368529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6eb8102230 0x7f6eb819f010 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.047+0000 7f6ebe67b640 1 --2- 172.21.15.78:0/993368529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6eb8101870 0x7f6eb8197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.047+0000 7f6ebe67b640 1 --2- 172.21.15.78:0/993368529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb8100680 0x7f6eb8197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.047+0000 7f6ebe67b640 1 -- 172.21.15.78:0/993368529 >> 172.21.15.78:0/993368529 conn(0x7f6eb80fbbf0 msgr2=0x7f6eb80688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:15.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.047+0000 7f6ebe67b640 1 -- 172.21.15.78:0/993368529 shutdown_connections 2024-04-25T01:43:15.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.047+0000 7f6ebe67b640 1 -- 172.21.15.78:0/993368529 wait complete. 2024-04-25T01:43:15.050 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T01:43:15.050 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T01:43:15.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.174+0000 7fe2d6207640 1 -- 172.21.15.78:0/951829920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d00ffa60 msgr2=0x7fe2d00ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:15.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.174+0000 7fe2d6207640 1 --2- 172.21.15.78:0/951829920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d00ffa60 0x7fe2d00ffe80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009930 tx=0x7fe2c402a420 comp rx=0 tx=0).stop 2024-04-25T01:43:15.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 -- 172.21.15.78:0/951829920 shutdown_connections 2024-04-25T01:43:15.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 --2- 172.21.15.78:0/951829920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d0103490 0x7fe2d0109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 --2- 172.21.15.78:0/951829920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2d0102ad0 0x7fe2d0102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 --2- 172.21.15.78:0/951829920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d00ffa60 0x7fe2d00ffe80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 -- 172.21.15.78:0/951829920 >> 172.21.15.78:0/951829920 conn(0x7fe2d00fb370 msgr2=0x7fe2d00fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:15.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 -- 172.21.15.78:0/951829920 shutdown_connections 2024-04-25T01:43:15.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 -- 172.21.15.78:0/951829920 wait complete. 2024-04-25T01:43:15.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 Processor -- start 2024-04-25T01:43:15.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 -- start start 2024-04-25T01:43:15.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d00ffa60 0x7fe2d0197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:15.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d0102ad0 0x7fe2d0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:15.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2d0103490 0x7fe2d019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:15.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d010d410 con 0x7fe2d0102ad0 2024-04-25T01:43:15.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d010d290 con 0x7fe2d00ffa60 2024-04-25T01:43:15.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.175+0000 7fe2d6207640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d010d590 con 0x7fe2d0103490 2024-04-25T01:43:15.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d0102ad0 0x7fe2d0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:15.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d0102ad0 0x7fe2d0198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56934/0 (socket says 172.21.15.78:56934) 2024-04-25T01:43:15.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2ceffd640 1 -- 172.21.15.78:0/1237003850 learned_addr learned my addr 172.21.15.78:0/1237003850 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:15.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2cffff640 1 --2- 172.21.15.78:0/1237003850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2d0103490 0x7fe2d019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:15.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2cf7fe640 1 --2- 172.21.15.78:0/1237003850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d00ffa60 0x7fe2d0197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:15.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2ceffd640 1 -- 172.21.15.78:0/1237003850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2d0103490 msgr2=0x7fe2d019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:15.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2ceffd640 1 --2- 172.21.15.78:0/1237003850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2d0103490 0x7fe2d019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2ceffd640 1 -- 172.21.15.78:0/1237003850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d00ffa60 msgr2=0x7fe2d0197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:15.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2ceffd640 1 --2- 172.21.15.78:0/1237003850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d00ffa60 0x7fe2d0197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2ceffd640 1 -- 172.21.15.78:0/1237003850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d019f750 con 0x7fe2d0102ad0 2024-04-25T01:43:15.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2ceffd640 1 --2- 172.21.15.78:0/1237003850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d0102ad0 0x7fe2d0198070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000e9f0 tx=0x7fe2c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:15.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2ccff9640 1 -- 172.21.15.78:0/1237003850 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000cd80 con 0x7fe2d0102ad0 2024-04-25T01:43:15.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2d6207640 1 -- 172.21.15.78:0/1237003850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d019fa40 con 0x7fe2d0102ad0 2024-04-25T01:43:15.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2ccff9640 1 -- 172.21.15.78:0/1237003850 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2c0005500 con 0x7fe2d0102ad0 2024-04-25T01:43:15.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2ccff9640 1 -- 172.21.15.78:0/1237003850 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0010690 con 0x7fe2d0102ad0 2024-04-25T01:43:15.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.176+0000 7fe2d6207640 1 -- 172.21.15.78:0/1237003850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d019ff90 con 0x7fe2d0102ad0 2024-04-25T01:43:15.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.177+0000 7fe2d6207640 1 -- 172.21.15.78:0/1237003850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d0108d20 con 0x7fe2d0102ad0 2024-04-25T01:43:15.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.178+0000 7fe2ccff9640 1 -- 172.21.15.78:0/1237003850 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe2c00046c0 con 0x7fe2d0102ad0 2024-04-25T01:43:15.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.179+0000 7fe2ccff9640 1 --2- 172.21.15.78:0/1237003850 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe2b80a2b90 0x7fe2b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:15.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.179+0000 7fe2cf7fe640 1 --2- 172.21.15.78:0/1237003850 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe2b80a2b90 0x7fe2b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:15.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.179+0000 7fe2ccff9640 1 -- 172.21.15.78:0/1237003850 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe2c00cf260 con 0x7fe2d0102ad0 2024-04-25T01:43:15.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.179+0000 7fe2cf7fe640 1 --2- 172.21.15.78:0/1237003850 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe2b80a2b90 0x7fe2b80a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009900 tx=0x7fe2c4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:15.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.186+0000 7fe2ccff9640 1 -- 172.21.15.78:0/1237003850 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe2c0053740 con 0x7fe2d0102ad0 2024-04-25T01:43:15.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:14 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:14 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:14 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:14 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:14 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1437801923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:15.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:14 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2206806990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:15.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:14 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:15.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:14 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:43:15.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:14 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:14 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:43:15.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:14 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3249522139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:43:15.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:15.369+0000 7fe2d6207640 1 -- 172.21.15.78:0/1237003850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe2d002d330 con 0x7fe2d0102ad0 2024-04-25T01:43:15.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:14 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1437801923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:15.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:14 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2206806990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:15.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:15.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:43:15.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:15.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:14 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:43:15.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:14 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3249522139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:43:15.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.004+0000 7fe2ccff9640 1 -- 172.21.15.78:0/1237003850 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7fe2c009c660 con 0x7fe2d0102ad0 2024-04-25T01:43:15.995 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-04-25T01:43:15.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.007+0000 7fe2d6207640 1 -- 172.21.15.78:0/1237003850 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe2b80a2b90 msgr2=0x7fe2b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:15.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.007+0000 7fe2d6207640 1 --2- 172.21.15.78:0/1237003850 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe2b80a2b90 0x7fe2b80a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009900 tx=0x7fe2c4000bc0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.007+0000 7fe2d6207640 1 -- 172.21.15.78:0/1237003850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d0102ad0 msgr2=0x7fe2d0198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:15.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.007+0000 7fe2d6207640 1 --2- 172.21.15.78:0/1237003850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d0102ad0 0x7fe2d0198070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000e9f0 tx=0x7fe2c000edf0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.007+0000 7fe2d6207640 1 -- 172.21.15.78:0/1237003850 shutdown_connections 2024-04-25T01:43:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.007+0000 7fe2d6207640 1 --2- 172.21.15.78:0/1237003850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2d0103490 0x7fe2d019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.007+0000 7fe2d6207640 1 --2- 172.21.15.78:0/1237003850 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe2b80a2b90 0x7fe2b80a5050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.007+0000 7fe2d6207640 1 --2- 172.21.15.78:0/1237003850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2d0102ad0 0x7fe2d0198070 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.007+0000 7fe2d6207640 1 --2- 172.21.15.78:0/1237003850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d00ffa60 0x7fe2d0197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.007+0000 7fe2d6207640 1 -- 172.21.15.78:0/1237003850 >> 172.21.15.78:0/1237003850 conn(0x7fe2d00fb370 msgr2=0x7fe2d0105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.007+0000 7fe2d6207640 1 -- 172.21.15.78:0/1237003850 shutdown_connections 2024-04-25T01:43:15.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.007+0000 7fe2d6207640 1 -- 172.21.15.78:0/1237003850 wait complete. 2024-04-25T01:43:16.013 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T01:43:16.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.143+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3216157247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d50100550 msgr2=0x7f5d50100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:16.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.143+0000 7f5d4f59e640 1 --2- 172.21.15.78:0/3216157247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d50100550 0x7f5d50100970 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009900 tx=0x7f5d4002a410 comp rx=0 tx=0).stop 2024-04-25T01:43:16.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3216157247 shutdown_connections 2024-04-25T01:43:16.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 --2- 172.21.15.78:0/3216157247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d50102100 0x7f5d50108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:16.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 --2- 172.21.15.78:0/3216157247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d50101740 0x7f5d50101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:16.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 --2- 172.21.15.78:0/3216157247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d50100550 0x7f5d50100970 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:16.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3216157247 >> 172.21.15.78:0/3216157247 conn(0x7f5d500fbae0 msgr2=0x7f5d500fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:16.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3216157247 shutdown_connections 2024-04-25T01:43:16.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3216157247 wait complete. 2024-04-25T01:43:16.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 Processor -- start 2024-04-25T01:43:16.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 -- start start 2024-04-25T01:43:16.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d50100550 0x7f5d50069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:16.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d50101740 0x7f5d5019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:16.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d50102100 0x7f5d5019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:16.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5010c230 con 0x7f5d50101740 2024-04-25T01:43:16.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5010c0b0 con 0x7f5d50100550 2024-04-25T01:43:16.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.144+0000 7f5d4f59e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5010c3b0 con 0x7f5d50102100 2024-04-25T01:43:16.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d47fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d50101740 0x7f5d5019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:16.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d47fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d50101740 0x7f5d5019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56942/0 (socket says 172.21.15.78:56942) 2024-04-25T01:43:16.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d47fff640 1 -- 172.21.15.78:0/3253966507 learned_addr learned my addr 172.21.15.78:0/3253966507 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:16.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d4e59c640 1 --2- 172.21.15.78:0/3253966507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d50100550 0x7f5d50069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:16.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d4ed9d640 1 --2- 172.21.15.78:0/3253966507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d50102100 0x7f5d5019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:16.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d47fff640 1 -- 172.21.15.78:0/3253966507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d50102100 msgr2=0x7f5d5019ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:16.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d47fff640 1 --2- 172.21.15.78:0/3253966507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d50102100 0x7f5d5019ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:16.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d47fff640 1 -- 172.21.15.78:0/3253966507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d50100550 msgr2=0x7f5d50069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:16.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d47fff640 1 --2- 172.21.15.78:0/3253966507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d50100550 0x7f5d50069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:16.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d47fff640 1 -- 172.21.15.78:0/3253966507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5019f5a0 con 0x7f5d50101740 2024-04-25T01:43:16.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d47fff640 1 --2- 172.21.15.78:0/3253966507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d50101740 0x7f5d5019c670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800d950 tx=0x7f5d3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:16.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d477fe640 1 -- 172.21.15.78:0/3253966507 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38017030 con 0x7f5d50101740 2024-04-25T01:43:16.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3253966507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5019f830 con 0x7f5d50101740 2024-04-25T01:43:16.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d477fe640 1 -- 172.21.15.78:0/3253966507 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d38004020 con 0x7f5d50101740 2024-04-25T01:43:16.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d477fe640 1 -- 172.21.15.78:0/3253966507 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d380047a0 con 0x7f5d50101740 2024-04-25T01:43:16.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.145+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3253966507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d5019fdd0 con 0x7f5d50101740 2024-04-25T01:43:16.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.146+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3253966507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d50107810 con 0x7f5d50101740 2024-04-25T01:43:16.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.148+0000 7f5d477fe640 1 -- 172.21.15.78:0/3253966507 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5d380041d0 con 0x7f5d50101740 2024-04-25T01:43:16.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.148+0000 7f5d477fe640 1 --2- 172.21.15.78:0/3253966507 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d3c0a2b90 0x7f5d3c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:16.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.148+0000 7f5d4e59c640 1 --2- 172.21.15.78:0/3253966507 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d3c0a2b90 0x7f5d3c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:16.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.149+0000 7f5d477fe640 1 -- 172.21.15.78:0/3253966507 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f5d380ced60 con 0x7f5d50101740 2024-04-25T01:43:16.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.149+0000 7f5d4e59c640 1 --2- 172.21.15.78:0/3253966507 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d3c0a2b90 0x7f5d3c0a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d4002aa00 tx=0x7f5d400315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:16.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.152+0000 7f5d477fe640 1 -- 172.21.15.78:0/3253966507 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5d3809bff0 con 0x7f5d50101740 2024-04-25T01:43:16.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:15 smithi078 ceph-mon[29469]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:16.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:15 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/993368529' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T01:43:16.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:15 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1237003850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T01:43:16.296 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:15 smithi152 ceph-mon[32465]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:16.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:15 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/993368529' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T01:43:16.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:15 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1237003850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T01:43:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:16.363+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3253966507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f5d5002d170 con 0x7f5d50101740 2024-04-25T01:43:16.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:16 smithi196 ceph-mon[32978]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:16.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:16 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/993368529' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T01:43:16.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:16 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1237003850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T01:43:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.008+0000 7f5d477fe640 1 -- 172.21.15.78:0/3253966507 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7f5d3809b8f0 con 0x7f5d50101740 2024-04-25T01:43:16.999 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-04-25T01:43:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.010+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3253966507 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d3c0a2b90 msgr2=0x7f5d3c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.010+0000 7f5d4f59e640 1 --2- 172.21.15.78:0/3253966507 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d3c0a2b90 0x7f5d3c0a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d4002aa00 tx=0x7f5d400315b0 comp rx=0 tx=0).stop 2024-04-25T01:43:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.010+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3253966507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d50101740 msgr2=0x7f5d5019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.010+0000 7f5d4f59e640 1 --2- 172.21.15.78:0/3253966507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d50101740 0x7f5d5019c670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800d950 tx=0x7f5d3800de10 comp rx=0 tx=0).stop 2024-04-25T01:43:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.010+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3253966507 shutdown_connections 2024-04-25T01:43:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.010+0000 7f5d4f59e640 1 --2- 172.21.15.78:0/3253966507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d50102100 0x7f5d5019ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.010+0000 7f5d4f59e640 1 --2- 172.21.15.78:0/3253966507 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d3c0a2b90 0x7f5d3c0a5050 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.010+0000 7f5d4f59e640 1 --2- 172.21.15.78:0/3253966507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d50101740 0x7f5d5019c670 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.010+0000 7f5d4f59e640 1 --2- 172.21.15.78:0/3253966507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d50100550 0x7f5d50069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.010+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3253966507 >> 172.21.15.78:0/3253966507 conn(0x7f5d500fbae0 msgr2=0x7f5d501049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.011+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3253966507 shutdown_connections 2024-04-25T01:43:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.011+0000 7f5d4f59e640 1 -- 172.21.15.78:0/3253966507 wait complete. 2024-04-25T01:43:17.021 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T01:43:17.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.133+0000 7f613d49a640 1 -- 172.21.15.78:0/166108129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6138100680 msgr2=0x7f6138100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:17.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.133+0000 7f613d49a640 1 --2- 172.21.15.78:0/166108129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6138100680 0x7f6138100aa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6128009900 tx=0x7f612802a410 comp rx=0 tx=0).stop 2024-04-25T01:43:17.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.133+0000 7f613d49a640 1 -- 172.21.15.78:0/166108129 shutdown_connections 2024-04-25T01:43:17.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.133+0000 7f613d49a640 1 --2- 172.21.15.78:0/166108129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6138102230 0x7f6138108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:17.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.133+0000 7f613d49a640 1 --2- 172.21.15.78:0/166108129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6138101870 0x7f6138101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:17.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.133+0000 7f613d49a640 1 --2- 172.21.15.78:0/166108129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6138100680 0x7f6138100aa0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:17.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.133+0000 7f613d49a640 1 -- 172.21.15.78:0/166108129 >> 172.21.15.78:0/166108129 conn(0x7f61380fbbf0 msgr2=0x7f61380fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:17.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.133+0000 7f613d49a640 1 -- 172.21.15.78:0/166108129 shutdown_connections 2024-04-25T01:43:17.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.133+0000 7f613d49a640 1 -- 172.21.15.78:0/166108129 wait complete. 2024-04-25T01:43:17.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.133+0000 7f613d49a640 1 Processor -- start 2024-04-25T01:43:17.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f613d49a640 1 -- start start 2024-04-25T01:43:17.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f613d49a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6138100680 0x7f6138197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:17.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f613d49a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6138101870 0x7f6138197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:17.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f613d49a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6138102230 0x7f613819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:17.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f613d49a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f613810c2d0 con 0x7f6138101870 2024-04-25T01:43:17.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f613d49a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f613810c150 con 0x7f6138100680 2024-04-25T01:43:17.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f613d49a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f613810c450 con 0x7f6138102230 2024-04-25T01:43:17.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f612e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6138101870 0x7f6138197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:17.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f612e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6138101870 0x7f6138197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56966/0 (socket says 172.21.15.78:56966) 2024-04-25T01:43:17.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f612e7fc640 1 -- 172.21.15.78:0/3589222263 learned_addr learned my addr 172.21.15.78:0/3589222263 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:17.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f6136ffd640 1 --2- 172.21.15.78:0/3589222263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6138100680 0x7f6138197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:17.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f61377fe640 1 --2- 172.21.15.78:0/3589222263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6138102230 0x7f613819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:17.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f612e7fc640 1 -- 172.21.15.78:0/3589222263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6138102230 msgr2=0x7f613819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:17.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f612e7fc640 1 --2- 172.21.15.78:0/3589222263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6138102230 0x7f613819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:17.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f612e7fc640 1 -- 172.21.15.78:0/3589222263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6138100680 msgr2=0x7f6138197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:17.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f612e7fc640 1 --2- 172.21.15.78:0/3589222263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6138100680 0x7f6138197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:17.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f612e7fc640 1 -- 172.21.15.78:0/3589222263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613819f6a0 con 0x7f6138101870 2024-04-25T01:43:17.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f612e7fc640 1 --2- 172.21.15.78:0/3589222263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6138101870 0x7f6138197fc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f612000d950 tx=0x7f612000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:17.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f6134ff9640 1 -- 172.21.15.78:0/3589222263 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120017070 con 0x7f6138101870 2024-04-25T01:43:17.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f613d49a640 1 -- 172.21.15.78:0/3589222263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613819f990 con 0x7f6138101870 2024-04-25T01:43:17.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f6134ff9640 1 -- 172.21.15.78:0/3589222263 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6120009c30 con 0x7f6138101870 2024-04-25T01:43:17.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.134+0000 7f6134ff9640 1 -- 172.21.15.78:0/3589222263 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612000f6b0 con 0x7f6138101870 2024-04-25T01:43:17.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.135+0000 7f613d49a640 1 -- 172.21.15.78:0/3589222263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613819fee0 con 0x7f6138101870 2024-04-25T01:43:17.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.136+0000 7f613d49a640 1 -- 172.21.15.78:0/3589222263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61381a0110 con 0x7f6138101870 2024-04-25T01:43:17.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.137+0000 7f6134ff9640 1 -- 172.21.15.78:0/3589222263 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f61200109d0 con 0x7f6138101870 2024-04-25T01:43:17.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.137+0000 7f6134ff9640 1 --2- 172.21.15.78:0/3589222263 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f61240a2b90 0x7f61240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:17.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.137+0000 7f6136ffd640 1 --2- 172.21.15.78:0/3589222263 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f61240a2b90 0x7f61240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:17.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.137+0000 7f6134ff9640 1 -- 172.21.15.78:0/3589222263 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f61200cea40 con 0x7f6138101870 2024-04-25T01:43:17.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.138+0000 7f6136ffd640 1 --2- 172.21.15.78:0/3589222263 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f61240a2b90 0x7f61240a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6128005ab0 tx=0x7f6128000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:17.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.141+0000 7f6134ff9640 1 -- 172.21.15.78:0/3589222263 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6120009de0 con 0x7f6138101870 2024-04-25T01:43:17.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:17 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1237003850' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T01:43:17.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:17 smithi152 ceph-mon[32465]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T01:43:17.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:17.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:43:17.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:17 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3253966507' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-25T01:43:17.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:17.353+0000 7f613d49a640 1 -- 172.21.15.78:0/3589222263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f61381a02e0 con 0x7f6138101870 2024-04-25T01:43:17.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:17 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1237003850' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T01:43:17.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:17 smithi196 ceph-mon[32978]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T01:43:17.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:17.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:43:17.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:17 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3253966507' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-25T01:43:17.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:17 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1237003850' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T01:43:17.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:17 smithi078 ceph-mon[29469]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T01:43:17.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:17.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:43:17.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:17.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:17 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3253966507' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-25T01:43:17.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:43:18.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.022+0000 7f6134ff9640 1 -- 172.21.15.78:0/3589222263 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f612009b4e0 con 0x7f6138101870 2024-04-25T01:43:18.013 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T01:43:18.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.025+0000 7f613d49a640 1 -- 172.21.15.78:0/3589222263 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f61240a2b90 msgr2=0x7f61240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:18.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.025+0000 7f613d49a640 1 --2- 172.21.15.78:0/3589222263 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f61240a2b90 0x7f61240a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6128005ab0 tx=0x7f6128000bc0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.025+0000 7f613d49a640 1 -- 172.21.15.78:0/3589222263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6138101870 msgr2=0x7f6138197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:18.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.025+0000 7f613d49a640 1 --2- 172.21.15.78:0/3589222263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6138101870 0x7f6138197fc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f612000d950 tx=0x7f612000de10 comp rx=0 tx=0).stop 2024-04-25T01:43:18.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.025+0000 7f613d49a640 1 -- 172.21.15.78:0/3589222263 shutdown_connections 2024-04-25T01:43:18.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.025+0000 7f613d49a640 1 --2- 172.21.15.78:0/3589222263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6138102230 0x7f613819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.025+0000 7f613d49a640 1 --2- 172.21.15.78:0/3589222263 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f61240a2b90 0x7f61240a5050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.025+0000 7f613d49a640 1 --2- 172.21.15.78:0/3589222263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6138101870 0x7f6138197fc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.025+0000 7f613d49a640 1 --2- 172.21.15.78:0/3589222263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6138100680 0x7f6138197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.025+0000 7f613d49a640 1 -- 172.21.15.78:0/3589222263 >> 172.21.15.78:0/3589222263 conn(0x7f61380fbbf0 msgr2=0x7f6138069d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:18.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.025+0000 7f613d49a640 1 -- 172.21.15.78:0/3589222263 shutdown_connections 2024-04-25T01:43:18.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.025+0000 7f613d49a640 1 -- 172.21.15.78:0/3589222263 wait complete. 2024-04-25T01:43:18.030 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:18 smithi078 ceph-mon[29469]: pgmap v283: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:18.030 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:18 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3253966507' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-25T01:43:18.030 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:18 smithi078 ceph-mon[29469]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T01:43:18.030 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:18 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3589222263' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T01:43:18.030 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:18 smithi078 ceph-mon[29469]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T01:43:18.030 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:18 smithi078 ceph-mon[29469]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T01:43:18.030 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:18 smithi078 ceph-mon[29469]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T01:43:18.031 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T01:43:18.032 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T01:43:18.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.147+0000 7f1f601e7640 1 -- 172.21.15.78:0/4054705750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f58102230 msgr2=0x7f1f58108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.147+0000 7f1f601e7640 1 --2- 172.21.15.78:0/4054705750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f58102230 0x7f1f58108ac0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009ca0 tx=0x7f1f5002a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.147+0000 7f1f601e7640 1 -- 172.21.15.78:0/4054705750 shutdown_connections 2024-04-25T01:43:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.147+0000 7f1f601e7640 1 --2- 172.21.15.78:0/4054705750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f58102230 0x7f1f58108ac0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.147+0000 7f1f601e7640 1 --2- 172.21.15.78:0/4054705750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f58101870 0x7f1f58101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.147+0000 7f1f601e7640 1 --2- 172.21.15.78:0/4054705750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f58100680 0x7f1f58100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.147+0000 7f1f601e7640 1 -- 172.21.15.78:0/4054705750 >> 172.21.15.78:0/4054705750 conn(0x7f1f580fbbf0 msgr2=0x7f1f580fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.147+0000 7f1f601e7640 1 -- 172.21.15.78:0/4054705750 shutdown_connections 2024-04-25T01:43:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.147+0000 7f1f601e7640 1 -- 172.21.15.78:0/4054705750 wait complete. 2024-04-25T01:43:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f601e7640 1 Processor -- start 2024-04-25T01:43:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f601e7640 1 -- start start 2024-04-25T01:43:18.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f601e7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f58100680 0x7f1f58197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f601e7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f58101870 0x7f1f58197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f601e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f58102230 0x7f1f5819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f601e7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5810c300 con 0x7f1f58102230 2024-04-25T01:43:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f601e7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5810c180 con 0x7f1f58100680 2024-04-25T01:43:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f601e7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5810c480 con 0x7f1f58101870 2024-04-25T01:43:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f5e784640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f58102230 0x7f1f5819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f5e784640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f58102230 0x7f1f5819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56978/0 (socket says 172.21.15.78:56978) 2024-04-25T01:43:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f5e784640 1 -- 172.21.15.78:0/693949268 learned_addr learned my addr 172.21.15.78:0/693949268 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f5d782640 1 --2- 172.21.15.78:0/693949268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f58101870 0x7f1f58197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f5df83640 1 --2- 172.21.15.78:0/693949268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f58100680 0x7f1f58197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:18.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f5e784640 1 -- 172.21.15.78:0/693949268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f58101870 msgr2=0x7f1f58197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f5e784640 1 --2- 172.21.15.78:0/693949268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f58101870 0x7f1f58197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f5e784640 1 -- 172.21.15.78:0/693949268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f58100680 msgr2=0x7f1f58197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f5e784640 1 --2- 172.21.15.78:0/693949268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f58100680 0x7f1f58197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f5e784640 1 -- 172.21.15.78:0/693949268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5819f680 con 0x7f1f58102230 2024-04-25T01:43:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.148+0000 7f1f5e784640 1 --2- 172.21.15.78:0/693949268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f58102230 0x7f1f5819f010 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1f50005e30 tx=0x7f1f50031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.149+0000 7f1f4effd640 1 -- 172.21.15.78:0/693949268 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50003c30 con 0x7f1f58102230 2024-04-25T01:43:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.149+0000 7f1f4effd640 1 -- 172.21.15.78:0/693949268 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f50003dd0 con 0x7f1f58102230 2024-04-25T01:43:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.149+0000 7f1f601e7640 1 -- 172.21.15.78:0/693949268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5819f910 con 0x7f1f58102230 2024-04-25T01:43:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.149+0000 7f1f4effd640 1 -- 172.21.15.78:0/693949268 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5002fde0 con 0x7f1f58102230 2024-04-25T01:43:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.149+0000 7f1f601e7640 1 -- 172.21.15.78:0/693949268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f5819fe60 con 0x7f1f58102230 2024-04-25T01:43:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.150+0000 7f1f601e7640 1 -- 172.21.15.78:0/693949268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f581a0030 con 0x7f1f58102230 2024-04-25T01:43:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.151+0000 7f1f4effd640 1 -- 172.21.15.78:0/693949268 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1f50004050 con 0x7f1f58102230 2024-04-25T01:43:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.152+0000 7f1f4effd640 1 --2- 172.21.15.78:0/693949268 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1f440a2b90 0x7f1f440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.152+0000 7f1f5df83640 1 --2- 172.21.15.78:0/693949268 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1f440a2b90 0x7f1f440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.152+0000 7f1f4effd640 1 -- 172.21.15.78:0/693949268 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1f500f0550 con 0x7f1f58102230 2024-04-25T01:43:18.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.152+0000 7f1f5df83640 1 --2- 172.21.15.78:0/693949268 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1f440a2b90 0x7f1f440a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1f54005fd0 tx=0x7f1f54005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:18.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.155+0000 7f1f4effd640 1 -- 172.21.15.78:0/693949268 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1f500bd6c0 con 0x7f1f58102230 2024-04-25T01:43:18.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:18 smithi152 ceph-mon[32465]: pgmap v283: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:18.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:18 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3253966507' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-25T01:43:18.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:18 smithi152 ceph-mon[32465]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T01:43:18.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:18 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3589222263' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T01:43:18.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:18 smithi152 ceph-mon[32465]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T01:43:18.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:18 smithi152 ceph-mon[32465]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T01:43:18.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:18 smithi152 ceph-mon[32465]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T01:43:18.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.341+0000 7f1f601e7640 1 -- 172.21.15.78:0/693949268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f1f581a0320 con 0x7f1f58102230 2024-04-25T01:43:18.364 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:18 smithi196 ceph-mon[32978]: pgmap v283: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:18.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:18 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3253966507' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-25T01:43:18.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:18 smithi196 ceph-mon[32978]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T01:43:18.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:18 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3589222263' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T01:43:18.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:18 smithi196 ceph-mon[32978]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T01:43:18.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:18 smithi196 ceph-mon[32978]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T01:43:18.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:18 smithi196 ceph-mon[32978]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T01:43:18.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.405+0000 7f1f4effd640 1 -- 172.21.15.78:0/693949268 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7f1f500bcfc0 con 0x7f1f58102230 2024-04-25T01:43:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.407+0000 7f1f601e7640 1 -- 172.21.15.78:0/693949268 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1f440a2b90 msgr2=0x7f1f440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.407+0000 7f1f601e7640 1 --2- 172.21.15.78:0/693949268 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1f440a2b90 0x7f1f440a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1f54005fd0 tx=0x7f1f54005850 comp rx=0 tx=0).stop 2024-04-25T01:43:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.407+0000 7f1f601e7640 1 -- 172.21.15.78:0/693949268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f58102230 msgr2=0x7f1f5819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.407+0000 7f1f601e7640 1 --2- 172.21.15.78:0/693949268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f58102230 0x7f1f5819f010 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1f50005e30 tx=0x7f1f50031ee0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.407+0000 7f1f601e7640 1 -- 172.21.15.78:0/693949268 shutdown_connections 2024-04-25T01:43:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.407+0000 7f1f601e7640 1 --2- 172.21.15.78:0/693949268 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1f440a2b90 0x7f1f440a5050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.407+0000 7f1f601e7640 1 --2- 172.21.15.78:0/693949268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f58102230 0x7f1f5819f010 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.407+0000 7f1f601e7640 1 --2- 172.21.15.78:0/693949268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f58101870 0x7f1f58197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.407+0000 7f1f601e7640 1 --2- 172.21.15.78:0/693949268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f58100680 0x7f1f58197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.407+0000 7f1f601e7640 1 -- 172.21.15.78:0/693949268 >> 172.21.15.78:0/693949268 conn(0x7f1f580fbbf0 msgr2=0x7f1f580fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.407+0000 7f1f601e7640 1 -- 172.21.15.78:0/693949268 shutdown_connections 2024-04-25T01:43:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.407+0000 7f1f601e7640 1 -- 172.21.15.78:0/693949268 wait complete. 2024-04-25T01:43:18.411 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T01:43:18.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.526+0000 7f96637b0640 1 -- 172.21.15.78:0/3651850375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f965c105570 msgr2=0x7f965c10be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:18.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.526+0000 7f96637b0640 1 --2- 172.21.15.78:0/3651850375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f965c105570 0x7f965c10be00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9654009ca0 tx=0x7f965402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.526+0000 7f96637b0640 1 -- 172.21.15.78:0/3651850375 shutdown_connections 2024-04-25T01:43:18.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.526+0000 7f96637b0640 1 --2- 172.21.15.78:0/3651850375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f965c105570 0x7f965c10be00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.526+0000 7f96637b0640 1 --2- 172.21.15.78:0/3651850375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f965c104bb0 0x7f965c105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.526+0000 7f96637b0640 1 --2- 172.21.15.78:0/3651850375 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f965c1039c0 0x7f965c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.526+0000 7f96637b0640 1 -- 172.21.15.78:0/3651850375 >> 172.21.15.78:0/3651850375 conn(0x7f965c0fef30 msgr2=0x7f965c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.526+0000 7f96637b0640 1 -- 172.21.15.78:0/3651850375 shutdown_connections 2024-04-25T01:43:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.526+0000 7f96637b0640 1 -- 172.21.15.78:0/3651850375 wait complete. 2024-04-25T01:43:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f96637b0640 1 Processor -- start 2024-04-25T01:43:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f96637b0640 1 -- start start 2024-04-25T01:43:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f96637b0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f965c1039c0 0x7f965c199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f96637b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f965c104bb0 0x7f965c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f96637b0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f965c105570 0x7f965c1a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f96637b0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f965c10f6a0 con 0x7f965c104bb0 2024-04-25T01:43:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f96637b0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f965c10f520 con 0x7f965c1039c0 2024-04-25T01:43:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f96637b0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f965c10f820 con 0x7f965c105570 2024-04-25T01:43:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f9660d4b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f965c104bb0 0x7f965c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f9660d4b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f965c104bb0 0x7f965c19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57000/0 (socket says 172.21.15.78:57000) 2024-04-25T01:43:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f9660d4b640 1 -- 172.21.15.78:0/1977168135 learned_addr learned my addr 172.21.15.78:0/1977168135 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f966154c640 1 --2- 172.21.15.78:0/1977168135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f965c1039c0 0x7f965c199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f9660d4b640 1 -- 172.21.15.78:0/1977168135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f965c105570 msgr2=0x7f965c1a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f9660d4b640 1 --2- 172.21.15.78:0/1977168135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f965c105570 0x7f965c1a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f9660d4b640 1 -- 172.21.15.78:0/1977168135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f965c1039c0 msgr2=0x7f965c199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f9660d4b640 1 --2- 172.21.15.78:0/1977168135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f965c1039c0 0x7f965c199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.527+0000 7f9660d4b640 1 -- 172.21.15.78:0/1977168135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965c1a1920 con 0x7f965c104bb0 2024-04-25T01:43:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.528+0000 7f9660d4b640 1 --2- 172.21.15.78:0/1977168135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f965c104bb0 0x7f965c19a250 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f964c00b4a0 tx=0x7f964c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.528+0000 7f96527fc640 1 -- 172.21.15.78:0/1977168135 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c014070 con 0x7f965c104bb0 2024-04-25T01:43:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.528+0000 7f96527fc640 1 -- 172.21.15.78:0/1977168135 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f964c010af0 con 0x7f965c104bb0 2024-04-25T01:43:18.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.528+0000 7f96637b0640 1 -- 172.21.15.78:0/1977168135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965c1a1c10 con 0x7f965c104bb0 2024-04-25T01:43:18.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.528+0000 7f96527fc640 1 -- 172.21.15.78:0/1977168135 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c017d10 con 0x7f965c104bb0 2024-04-25T01:43:18.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.528+0000 7f96637b0640 1 -- 172.21.15.78:0/1977168135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f965c1a2190 con 0x7f965c104bb0 2024-04-25T01:43:18.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.529+0000 7f96637b0640 1 -- 172.21.15.78:0/1977168135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f965c10ace0 con 0x7f965c104bb0 2024-04-25T01:43:18.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.530+0000 7f96527fc640 1 -- 172.21.15.78:0/1977168135 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f964c017070 con 0x7f965c104bb0 2024-04-25T01:43:18.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.530+0000 7f96527fc640 1 --2- 172.21.15.78:0/1977168135 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96480a2b90 0x7f96480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:18.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.530+0000 7f966154c640 1 --2- 172.21.15.78:0/1977168135 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96480a2b90 0x7f96480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:18.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.530+0000 7f96527fc640 1 -- 172.21.15.78:0/1977168135 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7f964c0ce580 con 0x7f965c104bb0 2024-04-25T01:43:18.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.531+0000 7f966154c640 1 --2- 172.21.15.78:0/1977168135 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96480a2b90 0x7f96480a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9658005db0 tx=0x7f9658005d40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:18.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.535+0000 7f96527fc640 1 -- 172.21.15.78:0/1977168135 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f964c09b4b0 con 0x7f965c104bb0 2024-04-25T01:43:18.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:18.711+0000 7f96637b0640 1 -- 172.21.15.78:0/1977168135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f965c02d220 con 0x7f965c104bb0 2024-04-25T01:43:19.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:19 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3589222263' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T01:43:19.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:19 smithi152 ceph-mon[32465]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T01:43:19.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:19 smithi152 ceph-mon[32465]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T01:43:19.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:19 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/693949268' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-25T01:43:19.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:19 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/693949268' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-25T01:43:19.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:19 smithi152 ceph-mon[32465]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T01:43:19.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:19 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1977168135' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T01:43:19.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:19 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3589222263' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T01:43:19.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:19 smithi196 ceph-mon[32978]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T01:43:19.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:19 smithi196 ceph-mon[32978]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T01:43:19.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:19 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/693949268' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-25T01:43:19.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:19 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/693949268' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-25T01:43:19.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:19 smithi196 ceph-mon[32978]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T01:43:19.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:19 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1977168135' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T01:43:19.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:19 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3589222263' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T01:43:19.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:19 smithi078 ceph-mon[29469]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T01:43:19.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:19 smithi078 ceph-mon[29469]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T01:43:19.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:19 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/693949268' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-25T01:43:19.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:19 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/693949268' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-25T01:43:19.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:19 smithi078 ceph-mon[29469]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T01:43:19.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:19 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1977168135' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T01:43:20.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:20 smithi152 ceph-mon[32465]: pgmap v287: 97 pgs: 29 active+clean, 6 creating+peering, 62 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:20.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:20 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] up:active 2024-04-25T01:43:20.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:20 smithi152 ceph-mon[32465]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:20.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:20 smithi152 ceph-mon[32465]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:20.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:20 smithi152 ceph-mon[32465]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T01:43:20.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:20 smithi152 ceph-mon[32465]: Cluster is now healthy 2024-04-25T01:43:20.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:20 smithi152 ceph-mon[32465]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T01:43:20.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:20 smithi196 ceph-mon[32978]: pgmap v287: 97 pgs: 29 active+clean, 6 creating+peering, 62 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:20.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:20 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] up:active 2024-04-25T01:43:20.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:20 smithi196 ceph-mon[32978]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:20.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:20 smithi196 ceph-mon[32978]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:20.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:20 smithi196 ceph-mon[32978]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T01:43:20.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:20 smithi196 ceph-mon[32978]: Cluster is now healthy 2024-04-25T01:43:20.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:20 smithi196 ceph-mon[32978]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T01:43:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.418+0000 7f96527fc640 1 -- 172.21.15.78:0/1977168135 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7f964c09adb0 con 0x7f965c104bb0 2024-04-25T01:43:20.408 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data_ec' created 2024-04-25T01:43:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.420+0000 7f96637b0640 1 -- 172.21.15.78:0/1977168135 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96480a2b90 msgr2=0x7f96480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.420+0000 7f96637b0640 1 --2- 172.21.15.78:0/1977168135 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96480a2b90 0x7f96480a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9658005db0 tx=0x7f9658005d40 comp rx=0 tx=0).stop 2024-04-25T01:43:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.420+0000 7f96637b0640 1 -- 172.21.15.78:0/1977168135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f965c104bb0 msgr2=0x7f965c19a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.420+0000 7f96637b0640 1 --2- 172.21.15.78:0/1977168135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f965c104bb0 0x7f965c19a250 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f964c00b4a0 tx=0x7f964c00b960 comp rx=0 tx=0).stop 2024-04-25T01:43:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.421+0000 7f96637b0640 1 -- 172.21.15.78:0/1977168135 shutdown_connections 2024-04-25T01:43:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.421+0000 7f96637b0640 1 --2- 172.21.15.78:0/1977168135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f965c105570 0x7f965c1a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:20.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.421+0000 7f96637b0640 1 --2- 172.21.15.78:0/1977168135 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96480a2b90 0x7f96480a5050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.421+0000 7f96637b0640 1 --2- 172.21.15.78:0/1977168135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f965c104bb0 0x7f965c19a250 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.421+0000 7f96637b0640 1 --2- 172.21.15.78:0/1977168135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f965c1039c0 0x7f965c199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.421+0000 7f96637b0640 1 -- 172.21.15.78:0/1977168135 >> 172.21.15.78:0/1977168135 conn(0x7f965c0fef30 msgr2=0x7f965c107eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.421+0000 7f96637b0640 1 -- 172.21.15.78:0/1977168135 shutdown_connections 2024-04-25T01:43:20.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.421+0000 7f96637b0640 1 -- 172.21.15.78:0/1977168135 wait complete. 2024-04-25T01:43:20.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:20 smithi078 ceph-mon[29469]: pgmap v287: 97 pgs: 29 active+clean, 6 creating+peering, 62 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:43:20.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:20 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] up:active 2024-04-25T01:43:20.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:20 smithi078 ceph-mon[29469]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:20.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:20 smithi078 ceph-mon[29469]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:20.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:20 smithi078 ceph-mon[29469]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T01:43:20.429 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:20 smithi078 ceph-mon[29469]: Cluster is now healthy 2024-04-25T01:43:20.430 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:20 smithi078 ceph-mon[29469]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T01:43:20.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T01:43:20.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.547+0000 7feaa889e640 1 -- 172.21.15.78:0/3009524785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaa40ff800 msgr2=0x7feaa40ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.547+0000 7feaa889e640 1 --2- 172.21.15.78:0/3009524785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaa40ff800 0x7feaa40ffc80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fea98009900 tx=0x7fea9802a410 comp rx=0 tx=0).stop 2024-04-25T01:43:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.547+0000 7feaa889e640 1 -- 172.21.15.78:0/3009524785 shutdown_connections 2024-04-25T01:43:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.547+0000 7feaa889e640 1 --2- 172.21.15.78:0/3009524785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaa41001c0 0x7feaa4109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.547+0000 7feaa889e640 1 --2- 172.21.15.78:0/3009524785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaa40ff800 0x7feaa40ffc80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.547+0000 7feaa889e640 1 --2- 172.21.15.78:0/3009524785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feaa4102d90 0x7feaa41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.547+0000 7feaa889e640 1 -- 172.21.15.78:0/3009524785 >> 172.21.15.78:0/3009524785 conn(0x7feaa40fb380 msgr2=0x7feaa40fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.547+0000 7feaa889e640 1 -- 172.21.15.78:0/3009524785 shutdown_connections 2024-04-25T01:43:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.547+0000 7feaa889e640 1 -- 172.21.15.78:0/3009524785 wait complete. 2024-04-25T01:43:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa889e640 1 Processor -- start 2024-04-25T01:43:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa889e640 1 -- start start 2024-04-25T01:43:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa889e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feaa40ff800 0x7feaa4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:20.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa889e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaa41001c0 0x7feaa4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa889e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaa4102d90 0x7feaa419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa889e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feaa410d4e0 con 0x7feaa41001c0 2024-04-25T01:43:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa889e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7feaa410d360 con 0x7feaa4102d90 2024-04-25T01:43:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa889e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feaa410d660 con 0x7feaa40ff800 2024-04-25T01:43:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa1d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaa41001c0 0x7feaa4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa1d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaa41001c0 0x7feaa4198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38574/0 (socket says 172.21.15.78:38574) 2024-04-25T01:43:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa1d9b640 1 -- 172.21.15.78:0/1051801569 learned_addr learned my addr 172.21.15.78:0/1051801569 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa259c640 1 --2- 172.21.15.78:0/1051801569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feaa40ff800 0x7feaa4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa2d9d640 1 --2- 172.21.15.78:0/1051801569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaa4102d90 0x7feaa419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:20.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa1d9b640 1 -- 172.21.15.78:0/1051801569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feaa40ff800 msgr2=0x7feaa4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa1d9b640 1 --2- 172.21.15.78:0/1051801569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feaa40ff800 0x7feaa4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa1d9b640 1 -- 172.21.15.78:0/1051801569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaa4102d90 msgr2=0x7feaa419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa1d9b640 1 --2- 172.21.15.78:0/1051801569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaa4102d90 0x7feaa419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.548+0000 7feaa1d9b640 1 -- 172.21.15.78:0/1051801569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa419f6e0 con 0x7feaa41001c0 2024-04-25T01:43:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.549+0000 7feaa1d9b640 1 --2- 172.21.15.78:0/1051801569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaa41001c0 0x7feaa4198000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fea9802aa00 tx=0x7fea98032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.549+0000 7fea937fe640 1 -- 172.21.15.78:0/1051801569 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9803f030 con 0x7feaa41001c0 2024-04-25T01:43:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.549+0000 7feaa889e640 1 -- 172.21.15.78:0/1051801569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa419f970 con 0x7feaa41001c0 2024-04-25T01:43:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.549+0000 7fea937fe640 1 -- 172.21.15.78:0/1051801569 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea98003ce0 con 0x7feaa41001c0 2024-04-25T01:43:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.549+0000 7fea937fe640 1 -- 172.21.15.78:0/1051801569 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98035070 con 0x7feaa41001c0 2024-04-25T01:43:20.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.549+0000 7feaa889e640 1 -- 172.21.15.78:0/1051801569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa419ff30 con 0x7feaa41001c0 2024-04-25T01:43:20.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.550+0000 7feaa889e640 1 -- 172.21.15.78:0/1051801569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa4108c50 con 0x7feaa41001c0 2024-04-25T01:43:20.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.551+0000 7fea937fe640 1 -- 172.21.15.78:0/1051801569 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fea98042030 con 0x7feaa41001c0 2024-04-25T01:43:20.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.551+0000 7fea937fe640 1 --2- 172.21.15.78:0/1051801569 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea8c0a2b90 0x7fea8c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:20.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.552+0000 7feaa259c640 1 --2- 172.21.15.78:0/1051801569 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea8c0a2b90 0x7fea8c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:20.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.552+0000 7fea937fe640 1 -- 172.21.15.78:0/1051801569 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7fea980efdc0 con 0x7feaa41001c0 2024-04-25T01:43:20.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.552+0000 7feaa259c640 1 --2- 172.21.15.78:0/1051801569 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea8c0a2b90 0x7fea8c0a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feaa4069a60 tx=0x7fea94009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:20.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.555+0000 7fea937fe640 1 -- 172.21.15.78:0/1051801569 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fea980bcb00 con 0x7feaa41001c0 2024-04-25T01:43:20.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:20.730+0000 7feaa889e640 1 -- 172.21.15.78:0/1051801569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7feaa40623b0 con 0x7feaa41001c0 2024-04-25T01:43:21.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.423+0000 7fea937fe640 1 -- 172.21.15.78:0/1051801569 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7fea980bc400 con 0x7feaa41001c0 2024-04-25T01:43:21.414 INFO:teuthology.orchestra.run.smithi078.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T01:43:21.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.427+0000 7feaa889e640 1 -- 172.21.15.78:0/1051801569 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea8c0a2b90 msgr2=0x7fea8c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:21.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.427+0000 7feaa889e640 1 --2- 172.21.15.78:0/1051801569 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea8c0a2b90 0x7fea8c0a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feaa4069a60 tx=0x7fea94009110 comp rx=0 tx=0).stop 2024-04-25T01:43:21.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.427+0000 7feaa889e640 1 -- 172.21.15.78:0/1051801569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaa41001c0 msgr2=0x7feaa4198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:21.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.427+0000 7feaa889e640 1 --2- 172.21.15.78:0/1051801569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaa41001c0 0x7feaa4198000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fea9802aa00 tx=0x7fea98032da0 comp rx=0 tx=0).stop 2024-04-25T01:43:21.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.428+0000 7feaa889e640 1 -- 172.21.15.78:0/1051801569 shutdown_connections 2024-04-25T01:43:21.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.428+0000 7feaa889e640 1 --2- 172.21.15.78:0/1051801569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaa4102d90 0x7feaa419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:21.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.428+0000 7feaa889e640 1 --2- 172.21.15.78:0/1051801569 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea8c0a2b90 0x7fea8c0a5050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:21.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.428+0000 7feaa889e640 1 --2- 172.21.15.78:0/1051801569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feaa41001c0 0x7feaa4198000 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:21.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.428+0000 7feaa889e640 1 --2- 172.21.15.78:0/1051801569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feaa40ff800 0x7feaa4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:21.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.428+0000 7feaa889e640 1 -- 172.21.15.78:0/1051801569 >> 172.21.15.78:0/1051801569 conn(0x7feaa40fb380 msgr2=0x7feaa4105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:21.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.428+0000 7feaa889e640 1 -- 172.21.15.78:0/1051801569 shutdown_connections 2024-04-25T01:43:21.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.428+0000 7feaa889e640 1 -- 172.21.15.78:0/1051801569 wait complete. 2024-04-25T01:43:21.441 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T01:43:21.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.555+0000 7f6c89d6a640 1 -- 172.21.15.78:0/3739781721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84101c90 msgr2=0x7f6c841020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:21.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.555+0000 7f6c89d6a640 1 --2- 172.21.15.78:0/3739781721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84101c90 0x7f6c841020b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009930 tx=0x7f6c7402a420 comp rx=0 tx=0).stop 2024-04-25T01:43:21.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.555+0000 7f6c89d6a640 1 -- 172.21.15.78:0/3739781721 shutdown_connections 2024-04-25T01:43:21.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.555+0000 7f6c89d6a640 1 --2- 172.21.15.78:0/3739781721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c841012f0 0x7f6c84108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:21.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.555+0000 7f6c89d6a640 1 --2- 172.21.15.78:0/3739781721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c84100930 0x7f6c84100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:21.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.555+0000 7f6c89d6a640 1 --2- 172.21.15.78:0/3739781721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84101c90 0x7f6c841020b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:21.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.555+0000 7f6c89d6a640 1 -- 172.21.15.78:0/3739781721 >> 172.21.15.78:0/3739781721 conn(0x7f6c840fb380 msgr2=0x7f6c840fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:21.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.555+0000 7f6c89d6a640 1 -- 172.21.15.78:0/3739781721 shutdown_connections 2024-04-25T01:43:21.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.555+0000 7f6c89d6a640 1 -- 172.21.15.78:0/3739781721 wait complete. 2024-04-25T01:43:21.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.555+0000 7f6c89d6a640 1 Processor -- start 2024-04-25T01:43:21.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c89d6a640 1 -- start start 2024-04-25T01:43:21.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c89d6a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84100930 0x7f6c841046e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:21.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c89d6a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c841012f0 0x7f6c84102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:21.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c89d6a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c84101c90 0x7f6c84103260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:21.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c89d6a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8410c380 con 0x7f6c84100930 2024-04-25T01:43:21.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c89d6a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8410c200 con 0x7f6c841012f0 2024-04-25T01:43:21.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c89d6a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8410c500 con 0x7f6c84101c90 2024-04-25T01:43:21.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c837fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84100930 0x7f6c841046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:21.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c837fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84100930 0x7f6c841046e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38596/0 (socket says 172.21.15.78:38596) 2024-04-25T01:43:21.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c837fe640 1 -- 172.21.15.78:0/4060094538 learned_addr learned my addr 172.21.15.78:0/4060094538 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:21.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c82ffd640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c841012f0 0x7f6c84102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:21.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c83fff640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c84101c90 0x7f6c84103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:21.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c837fe640 1 -- 172.21.15.78:0/4060094538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c84101c90 msgr2=0x7f6c84103260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:21.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c837fe640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c84101c90 0x7f6c84103260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:21.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c837fe640 1 -- 172.21.15.78:0/4060094538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c841012f0 msgr2=0x7f6c84102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:21.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c837fe640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c841012f0 0x7f6c84102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:21.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c837fe640 1 -- 172.21.15.78:0/4060094538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c84103af0 con 0x7f6c84100930 2024-04-25T01:43:21.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c82ffd640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c841012f0 0x7f6c84102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:43:21.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c837fe640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84100930 0x7f6c841046e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6c74031440 tx=0x7f6c740043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:21.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c80ff9640 1 -- 172.21.15.78:0/4060094538 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7403f030 con 0x7f6c84100930 2024-04-25T01:43:21.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c89d6a640 1 -- 172.21.15.78:0/4060094538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c841a1ba0 con 0x7f6c84100930 2024-04-25T01:43:21.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c80ff9640 1 -- 172.21.15.78:0/4060094538 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c74003ce0 con 0x7f6c84100930 2024-04-25T01:43:21.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.556+0000 7f6c80ff9640 1 -- 172.21.15.78:0/4060094538 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74035070 con 0x7f6c84100930 2024-04-25T01:43:21.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.557+0000 7f6c89d6a640 1 -- 172.21.15.78:0/4060094538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c841a20f0 con 0x7f6c84100930 2024-04-25T01:43:21.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.557+0000 7f6c89d6a640 1 -- 172.21.15.78:0/4060094538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c841a22c0 con 0x7f6c84100930 2024-04-25T01:43:21.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.558+0000 7f6c80ff9640 1 -- 172.21.15.78:0/4060094538 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6c74042030 con 0x7f6c84100930 2024-04-25T01:43:21.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.559+0000 7f6c80ff9640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6c6c0a2b90 0x7f6c6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:21.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.559+0000 7f6c82ffd640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6c6c0a2b90 0x7f6c6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:21.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.559+0000 7f6c80ff9640 1 -- 172.21.15.78:0/4060094538 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f6c740ef970 con 0x7f6c84100930 2024-04-25T01:43:21.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.559+0000 7f6c82ffd640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6c6c0a2b90 0x7f6c6c0a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6c84104450 tx=0x7f6c78005d90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:21.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.563+0000 7f6c80ff9640 1 -- 172.21.15.78:0/4060094538 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6c740bc6b0 con 0x7f6c84100930 2024-04-25T01:43:21.755 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:21 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1977168135' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T01:43:21.755 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:21 smithi078 ceph-mon[29469]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T01:43:21.755 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:21 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1051801569' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-25T01:43:21.755 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:21 smithi078 ceph-mon[29469]: pgmap v290: 129 pgs: 85 active+clean, 9 creating+peering, 35 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-04-25T01:43:21.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:21.780+0000 7f6c89d6a640 1 -- 172.21.15.78:0/4060094538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f6c841a26b0 con 0x7f6c84100930 2024-04-25T01:43:21.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:21 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1977168135' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T01:43:21.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:21 smithi152 ceph-mon[32465]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T01:43:21.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:21 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1051801569' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-25T01:43:21.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:21 smithi152 ceph-mon[32465]: pgmap v290: 129 pgs: 85 active+clean, 9 creating+peering, 35 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-04-25T01:43:21.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:21 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1977168135' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T01:43:21.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:21 smithi196 ceph-mon[32978]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T01:43:21.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:21 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1051801569' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-25T01:43:21.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:21 smithi196 ceph-mon[32978]: pgmap v290: 129 pgs: 85 active+clean, 9 creating+peering, 35 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-04-25T01:43:22.297 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:43:21 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-b[62804]: 2024-04-25T01:43:21.960+0000 7faf6a3c9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T01:43:22.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.425+0000 7f6c80ff9640 1 -- 172.21.15.78:0/4060094538 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f6c740bbfb0 con 0x7f6c84100930 2024-04-25T01:43:22.416 INFO:teuthology.orchestra.run.smithi078.stderr:added data pool 4 to fsmap 2024-04-25T01:43:22.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.427+0000 7f6c89d6a640 1 -- 172.21.15.78:0/4060094538 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6c6c0a2b90 msgr2=0x7f6c6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:22.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.427+0000 7f6c89d6a640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6c6c0a2b90 0x7f6c6c0a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6c84104450 tx=0x7f6c78005d90 comp rx=0 tx=0).stop 2024-04-25T01:43:22.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.427+0000 7f6c89d6a640 1 -- 172.21.15.78:0/4060094538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84100930 msgr2=0x7f6c841046e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:22.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.427+0000 7f6c89d6a640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84100930 0x7f6c841046e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6c74031440 tx=0x7f6c740043b0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.427+0000 7f6c89d6a640 1 -- 172.21.15.78:0/4060094538 shutdown_connections 2024-04-25T01:43:22.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.427+0000 7f6c89d6a640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c84101c90 0x7f6c84103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.427+0000 7f6c89d6a640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c841012f0 0x7f6c84102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.427+0000 7f6c89d6a640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6c6c0a2b90 0x7f6c6c0a5050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.427+0000 7f6c89d6a640 1 --2- 172.21.15.78:0/4060094538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84100930 0x7f6c841046e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.427+0000 7f6c89d6a640 1 -- 172.21.15.78:0/4060094538 >> 172.21.15.78:0/4060094538 conn(0x7f6c840fb380 msgr2=0x7f6c840fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:22.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.428+0000 7f6c89d6a640 1 -- 172.21.15.78:0/4060094538 shutdown_connections 2024-04-25T01:43:22.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.428+0000 7f6c89d6a640 1 -- 172.21.15.78:0/4060094538 wait complete. 2024-04-25T01:43:22.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:43:22.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.540+0000 7f1d5659e640 1 -- 172.21.15.78:0/3920386320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d50102230 msgr2=0x7f1d50108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:22.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.540+0000 7f1d5659e640 1 --2- 172.21.15.78:0/3920386320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d50102230 0x7f1d50108ac0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009ca0 tx=0x7f1d4802a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.540+0000 7f1d5659e640 1 -- 172.21.15.78:0/3920386320 shutdown_connections 2024-04-25T01:43:22.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.540+0000 7f1d5659e640 1 --2- 172.21.15.78:0/3920386320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d50102230 0x7f1d50108ac0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.540+0000 7f1d5659e640 1 --2- 172.21.15.78:0/3920386320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d50101870 0x7f1d50101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.540+0000 7f1d5659e640 1 --2- 172.21.15.78:0/3920386320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d50100680 0x7f1d50100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.540+0000 7f1d5659e640 1 -- 172.21.15.78:0/3920386320 >> 172.21.15.78:0/3920386320 conn(0x7f1d500fbbf0 msgr2=0x7f1d500fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.540+0000 7f1d5659e640 1 -- 172.21.15.78:0/3920386320 shutdown_connections 2024-04-25T01:43:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.540+0000 7f1d5659e640 1 -- 172.21.15.78:0/3920386320 wait complete. 2024-04-25T01:43:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.540+0000 7f1d5659e640 1 Processor -- start 2024-04-25T01:43:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.540+0000 7f1d5659e640 1 -- start start 2024-04-25T01:43:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d5659e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d50100680 0x7f1d50069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d5659e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d50101870 0x7f1d5019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d5659e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d50102230 0x7f1d5019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d5659e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5010c3f0 con 0x7f1d50101870 2024-04-25T01:43:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d5659e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5010c270 con 0x7f1d50102230 2024-04-25T01:43:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d5659e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5010c570 con 0x7f1d50100680 2024-04-25T01:43:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d4f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d50101870 0x7f1d5019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d4f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d50101870 0x7f1d5019c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38622/0 (socket says 172.21.15.78:38622) 2024-04-25T01:43:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d4f7fe640 1 -- 172.21.15.78:0/351739492 learned_addr learned my addr 172.21.15.78:0/351739492 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d4ffff640 1 --2- 172.21.15.78:0/351739492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d50100680 0x7f1d50069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d4f7fe640 1 -- 172.21.15.78:0/351739492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d50100680 msgr2=0x7f1d50069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d4f7fe640 1 --2- 172.21.15.78:0/351739492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d50100680 0x7f1d50069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d4f7fe640 1 -- 172.21.15.78:0/351739492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d50102230 msgr2=0x7f1d5019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d4f7fe640 1 --2- 172.21.15.78:0/351739492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d50102230 0x7f1d5019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d4f7fe640 1 -- 172.21.15.78:0/351739492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5019f7d0 con 0x7f1d50101870 2024-04-25T01:43:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d4f7fe640 1 --2- 172.21.15.78:0/351739492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d50101870 0x7f1d5019c8d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000b4a0 tx=0x7f1d4000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d4d7fa640 1 -- 172.21.15.78:0/351739492 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40014070 con 0x7f1d50101870 2024-04-25T01:43:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d4d7fa640 1 -- 172.21.15.78:0/351739492 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d40010af0 con 0x7f1d50101870 2024-04-25T01:43:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d5659e640 1 -- 172.21.15.78:0/351739492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5019fac0 con 0x7f1d50101870 2024-04-25T01:43:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d4d7fa640 1 -- 172.21.15.78:0/351739492 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40017d10 con 0x7f1d50101870 2024-04-25T01:43:22.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.541+0000 7f1d5659e640 1 -- 172.21.15.78:0/351739492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d501a0090 con 0x7f1d50101870 2024-04-25T01:43:22.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.542+0000 7f1d5659e640 1 -- 172.21.15.78:0/351739492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d501079a0 con 0x7f1d50101870 2024-04-25T01:43:22.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.544+0000 7f1d4d7fa640 1 -- 172.21.15.78:0/351739492 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1d40017070 con 0x7f1d50101870 2024-04-25T01:43:22.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.545+0000 7f1d4d7fa640 1 --2- 172.21.15.78:0/351739492 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1d380a2b90 0x7f1d380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:22.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.545+0000 7f1d4ffff640 1 --2- 172.21.15.78:0/351739492 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1d380a2b90 0x7f1d380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:22.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.545+0000 7f1d4d7fa640 1 -- 172.21.15.78:0/351739492 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1d400cea90 con 0x7f1d50101870 2024-04-25T01:43:22.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.545+0000 7f1d4ffff640 1 --2- 172.21.15.78:0/351739492 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1d380a2b90 0x7f1d380a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1d501016d0 tx=0x7f1d44008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:22.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.548+0000 7f1d4d7fa640 1 -- 172.21.15.78:0/351739492 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1d4009b7b0 con 0x7f1d50101870 2024-04-25T01:43:22.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.761+0000 7f1d5659e640 1 -- 172.21.15.78:0/351739492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d5002d170 con 0x7f1d50101870 2024-04-25T01:43:22.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.762+0000 7f1d4d7fa640 1 -- 172.21.15.78:0/351739492 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11809 (secure 0 0 0) 0x7f1d4009b0b0 con 0x7f1d50101870 2024-04-25T01:43:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:22.753 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/1337519666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1337519666},{"type":"v1","addr":"172.21.15.78:6839","nonce":1337519666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1963410683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1963410683},{"type":"v1","addr":"172.21.15.78:6841","nonce":1963410683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:43:17.355192+0000","modified":"2024-04-25T01:43:21.781823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1969344843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1969344843},{"type":"v1","addr":"172.21.15.152:6839","nonce":1969344843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2024-04-25T01:43:22.753 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 17 2024-04-25T01:43:22.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.764+0000 7f1d5659e640 1 -- 172.21.15.78:0/351739492 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1d380a2b90 msgr2=0x7f1d380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:22.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.764+0000 7f1d5659e640 1 --2- 172.21.15.78:0/351739492 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1d380a2b90 0x7f1d380a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1d501016d0 tx=0x7f1d44008040 comp rx=0 tx=0).stop 2024-04-25T01:43:22.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.764+0000 7f1d5659e640 1 -- 172.21.15.78:0/351739492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d50101870 msgr2=0x7f1d5019c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:22.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.764+0000 7f1d5659e640 1 --2- 172.21.15.78:0/351739492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d50101870 0x7f1d5019c8d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000b4a0 tx=0x7f1d4000b960 comp rx=0 tx=0).stop 2024-04-25T01:43:22.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.764+0000 7f1d5659e640 1 -- 172.21.15.78:0/351739492 shutdown_connections 2024-04-25T01:43:22.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.764+0000 7f1d5659e640 1 --2- 172.21.15.78:0/351739492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d50102230 0x7f1d5019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.764+0000 7f1d5659e640 1 --2- 172.21.15.78:0/351739492 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1d380a2b90 0x7f1d380a5050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.764+0000 7f1d5659e640 1 --2- 172.21.15.78:0/351739492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d50101870 0x7f1d5019c8d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.764+0000 7f1d5659e640 1 --2- 172.21.15.78:0/351739492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d50100680 0x7f1d50069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.764+0000 7f1d5659e640 1 -- 172.21.15.78:0/351739492 >> 172.21.15.78:0/351739492 conn(0x7f1d500fbbf0 msgr2=0x7f1d50104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:22.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.764+0000 7f1d5659e640 1 -- 172.21.15.78:0/351739492 shutdown_connections 2024-04-25T01:43:22.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.764+0000 7f1d5659e640 1 -- 172.21.15.78:0/351739492 wait complete. 2024-04-25T01:43:22.756 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:22 smithi078 ceph-mon[29469]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T01:43:22.756 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:22 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1051801569' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-25T01:43:22.756 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:22 smithi078 ceph-mon[29469]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T01:43:22.756 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:22 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/4060094538' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-25T01:43:22.766 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:43:22.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:22 smithi152 ceph-mon[32465]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T01:43:22.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:22 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1051801569' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-25T01:43:22.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:22 smithi152 ceph-mon[32465]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T01:43:22.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:22 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/4060094538' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-25T01:43:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 -- 172.21.15.78:0/3809551504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6588104b40 msgr2=0x7f6588106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 --2- 172.21.15.78:0/3809551504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6588104b40 0x7f6588106f30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6578009900 tx=0x7f657802a410 comp rx=0 tx=0).stop 2024-04-25T01:43:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 -- 172.21.15.78:0/3809551504 shutdown_connections 2024-04-25T01:43:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 --2- 172.21.15.78:0/3809551504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6588107470 0x7f65881098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 --2- 172.21.15.78:0/3809551504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6588104b40 0x7f6588106f30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 --2- 172.21.15.78:0/3809551504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588102210 0x7f6588104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 -- 172.21.15.78:0/3809551504 >> 172.21.15.78:0/3809551504 conn(0x7f65880fbc40 msgr2=0x7f65880fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 -- 172.21.15.78:0/3809551504 shutdown_connections 2024-04-25T01:43:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 -- 172.21.15.78:0/3809551504 wait complete. 2024-04-25T01:43:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 Processor -- start 2024-04-25T01:43:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 -- start start 2024-04-25T01:43:22.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6588102210 0x7f6588197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6588104b40 0x7f6588198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588107470 0x7f658819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f658810d530 con 0x7f6588104b40 2024-04-25T01:43:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f658810d3b0 con 0x7f6588107470 2024-04-25T01:43:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f6590015640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f658810d6b0 con 0x7f6588102210 2024-04-25T01:43:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f658d5b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6588104b40 0x7f6588198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f658d5b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6588104b40 0x7f6588198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38626/0 (socket says 172.21.15.78:38626) 2024-04-25T01:43:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.872+0000 7f658d5b0640 1 -- 172.21.15.78:0/664122544 learned_addr learned my addr 172.21.15.78:0/664122544 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.873+0000 7f658ddb1640 1 --2- 172.21.15.78:0/664122544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6588102210 0x7f6588197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.873+0000 7f658d5b0640 1 -- 172.21.15.78:0/664122544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6588102210 msgr2=0x7f6588197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.873+0000 7f658e5b2640 1 --2- 172.21.15.78:0/664122544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588107470 0x7f658819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.873+0000 7f658d5b0640 1 --2- 172.21.15.78:0/664122544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6588102210 0x7f6588197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.873+0000 7f658d5b0640 1 -- 172.21.15.78:0/664122544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588107470 msgr2=0x7f658819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:22.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.873+0000 7f658d5b0640 1 --2- 172.21.15.78:0/664122544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588107470 0x7f658819f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:22.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.873+0000 7f658d5b0640 1 -- 172.21.15.78:0/664122544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658819f830 con 0x7f6588104b40 2024-04-25T01:43:22.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.873+0000 7f658d5b0640 1 --2- 172.21.15.78:0/664122544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6588104b40 0x7f6588198160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f657802aa00 tx=0x7f6578032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:22.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.873+0000 7f657effd640 1 -- 172.21.15.78:0/664122544 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657803f030 con 0x7f6588104b40 2024-04-25T01:43:22.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.873+0000 7f6590015640 1 -- 172.21.15.78:0/664122544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658819fac0 con 0x7f6588104b40 2024-04-25T01:43:22.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.873+0000 7f657effd640 1 -- 172.21.15.78:0/664122544 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6578003ce0 con 0x7f6588104b40 2024-04-25T01:43:22.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.873+0000 7f657effd640 1 -- 172.21.15.78:0/664122544 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578035070 con 0x7f6588104b40 2024-04-25T01:43:22.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.873+0000 7f6590015640 1 -- 172.21.15.78:0/664122544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65881a0030 con 0x7f6588104b40 2024-04-25T01:43:22.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.874+0000 7f6590015640 1 -- 172.21.15.78:0/664122544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65880fd730 con 0x7f6588104b40 2024-04-25T01:43:22.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:22 smithi196 ceph-mon[32978]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T01:43:22.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:22 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1051801569' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-25T01:43:22.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:22 smithi196 ceph-mon[32978]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T01:43:22.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:22 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/4060094538' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-25T01:43:22.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.876+0000 7f657effd640 1 -- 172.21.15.78:0/664122544 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6578042030 con 0x7f6588104b40 2024-04-25T01:43:22.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.876+0000 7f657effd640 1 --2- 172.21.15.78:0/664122544 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f65740a2b90 0x7f65740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:22.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.876+0000 7f658ddb1640 1 --2- 172.21.15.78:0/664122544 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f65740a2b90 0x7f65740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:22.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.876+0000 7f657effd640 1 -- 172.21.15.78:0/664122544 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f65780efee0 con 0x7f6588104b40 2024-04-25T01:43:22.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.877+0000 7f658ddb1640 1 --2- 172.21.15.78:0/664122544 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f65740a2b90 0x7f65740a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f65881990f0 tx=0x7f6580007310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:22.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:22.880+0000 7f657effd640 1 -- 172.21.15.78:0/664122544 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f65780bcc00 con 0x7f6588104b40 2024-04-25T01:43:23.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.058+0000 7f6590015640 1 -- 172.21.15.78:0/664122544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f658802d3c0 con 0x7f6588104b40 2024-04-25T01:43:23.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.060+0000 7f657effd640 1 -- 172.21.15.78:0/664122544 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24151 (secure 0 0 0) 0x7f65780bc500 con 0x7f6588104b40 2024-04-25T01:43:23.051 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:23.051 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:43:21.781660+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:43:23.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.062+0000 7f6590015640 1 -- 172.21.15.78:0/664122544 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f65740a2b90 msgr2=0x7f65740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.062+0000 7f6590015640 1 --2- 172.21.15.78:0/664122544 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f65740a2b90 0x7f65740a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f65881990f0 tx=0x7f6580007310 comp rx=0 tx=0).stop 2024-04-25T01:43:23.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.062+0000 7f6590015640 1 -- 172.21.15.78:0/664122544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6588104b40 msgr2=0x7f6588198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.062+0000 7f6590015640 1 --2- 172.21.15.78:0/664122544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6588104b40 0x7f6588198160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f657802aa00 tx=0x7f6578032da0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.062+0000 7f6590015640 1 -- 172.21.15.78:0/664122544 shutdown_connections 2024-04-25T01:43:23.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.062+0000 7f6590015640 1 --2- 172.21.15.78:0/664122544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6588107470 0x7f658819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.062+0000 7f6590015640 1 --2- 172.21.15.78:0/664122544 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f65740a2b90 0x7f65740a5050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.062+0000 7f6590015640 1 --2- 172.21.15.78:0/664122544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6588104b40 0x7f6588198160 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.062+0000 7f6590015640 1 --2- 172.21.15.78:0/664122544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6588102210 0x7f6588197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.062+0000 7f6590015640 1 -- 172.21.15.78:0/664122544 >> 172.21.15.78:0/664122544 conn(0x7f65880fbc40 msgr2=0x7f6588102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:23.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.062+0000 7f6590015640 1 -- 172.21.15.78:0/664122544 shutdown_connections 2024-04-25T01:43:23.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.062+0000 7f6590015640 1 -- 172.21.15.78:0/664122544 wait complete. 2024-04-25T01:43:23.064 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:43:23.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.174+0000 7f6c89c1c640 1 -- 172.21.15.78:0/569746545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84100680 msgr2=0x7f6c84100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.174+0000 7f6c89c1c640 1 --2- 172.21.15.78:0/569746545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84100680 0x7f6c84100aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009900 tx=0x7f6c7402a410 comp rx=0 tx=0).stop 2024-04-25T01:43:23.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 -- 172.21.15.78:0/569746545 shutdown_connections 2024-04-25T01:43:23.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 --2- 172.21.15.78:0/569746545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c84102230 0x7f6c84108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 --2- 172.21.15.78:0/569746545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c84101870 0x7f6c84101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 --2- 172.21.15.78:0/569746545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84100680 0x7f6c84100aa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 -- 172.21.15.78:0/569746545 >> 172.21.15.78:0/569746545 conn(0x7f6c840fbbf0 msgr2=0x7f6c840fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 -- 172.21.15.78:0/569746545 shutdown_connections 2024-04-25T01:43:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 -- 172.21.15.78:0/569746545 wait complete. 2024-04-25T01:43:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 Processor -- start 2024-04-25T01:43:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 -- start start 2024-04-25T01:43:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c84100680 0x7f6c84069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c84101870 0x7f6c8419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84102230 0x7f6c8419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8410c350 con 0x7f6c84102230 2024-04-25T01:43:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8410c1d0 con 0x7f6c84101870 2024-04-25T01:43:23.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c89c1c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8410c4d0 con 0x7f6c84100680 2024-04-25T01:43:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c83fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84102230 0x7f6c8419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c83fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84102230 0x7f6c8419ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38646/0 (socket says 172.21.15.78:38646) 2024-04-25T01:43:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.175+0000 7f6c83fff640 1 -- 172.21.15.78:0/4088345220 learned_addr learned my addr 172.21.15.78:0/4088345220 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.176+0000 7f6c837fe640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c84100680 0x7f6c84069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.176+0000 7f6c7bfff640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c84101870 0x7f6c8419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.176+0000 7f6c83fff640 1 -- 172.21.15.78:0/4088345220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c84100680 msgr2=0x7f6c84069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.176+0000 7f6c83fff640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c84100680 0x7f6c84069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.176+0000 7f6c83fff640 1 -- 172.21.15.78:0/4088345220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c84101870 msgr2=0x7f6c8419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.176+0000 7f6c83fff640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c84101870 0x7f6c8419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.176+0000 7f6c83fff640 1 -- 172.21.15.78:0/4088345220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8419f6f0 con 0x7f6c84102230 2024-04-25T01:43:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.176+0000 7f6c7bfff640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c84101870 0x7f6c8419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:43:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.176+0000 7f6c83fff640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84102230 0x7f6c8419ee30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00cce0 tx=0x7f6c7c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.176+0000 7f6c817fa640 1 -- 172.21.15.78:0/4088345220 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c00f450 con 0x7f6c84102230 2024-04-25T01:43:23.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.176+0000 7f6c89c1c640 1 -- 172.21.15.78:0/4088345220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8419f9e0 con 0x7f6c84102230 2024-04-25T01:43:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.176+0000 7f6c817fa640 1 -- 172.21.15.78:0/4088345220 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c7c00fad0 con 0x7f6c84102230 2024-04-25T01:43:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.176+0000 7f6c817fa640 1 -- 172.21.15.78:0/4088345220 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c018c10 con 0x7f6c84102230 2024-04-25T01:43:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.176+0000 7f6c89c1c640 1 -- 172.21.15.78:0/4088345220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c8419ff60 con 0x7f6c84102230 2024-04-25T01:43:23.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.177+0000 7f6c89c1c640 1 -- 172.21.15.78:0/4088345220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c841079a0 con 0x7f6c84102230 2024-04-25T01:43:23.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.178+0000 7f6c817fa640 1 -- 172.21.15.78:0/4088345220 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6c7c00f5f0 con 0x7f6c84102230 2024-04-25T01:43:23.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.178+0000 7f6c817fa640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6c700a2b90 0x7f6c700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:23.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.179+0000 7f6c837fe640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6c700a2b90 0x7f6c700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:23.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.179+0000 7f6c817fa640 1 -- 172.21.15.78:0/4088345220 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6c7c0cf9f0 con 0x7f6c84102230 2024-04-25T01:43:23.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.179+0000 7f6c837fe640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6c700a2b90 0x7f6c700a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c74006010 tx=0x7f6c74000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:23.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.183+0000 7f6c817fa640 1 -- 172.21.15.78:0/4088345220 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6c7c014030 con 0x7f6c84102230 2024-04-25T01:43:23.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.359+0000 7f6c89c1c640 1 -- 172.21.15.78:0/4088345220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c84062340 con 0x7f6c84102230 2024-04-25T01:43:23.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.360+0000 7f6c817fa640 1 -- 172.21.15.78:0/4088345220 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24151 (secure 0 0 0) 0x7f6c7c09c710 con 0x7f6c84102230 2024-04-25T01:43:23.351 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:23.351 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:43:21.781660+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:43:23.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.362+0000 7f6c89c1c640 1 -- 172.21.15.78:0/4088345220 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6c700a2b90 msgr2=0x7f6c700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.362+0000 7f6c89c1c640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6c700a2b90 0x7f6c700a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c74006010 tx=0x7f6c74000bc0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.362+0000 7f6c89c1c640 1 -- 172.21.15.78:0/4088345220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84102230 msgr2=0x7f6c8419ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.362+0000 7f6c89c1c640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84102230 0x7f6c8419ee30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00cce0 tx=0x7f6c7c00a620 comp rx=0 tx=0).stop 2024-04-25T01:43:23.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.362+0000 7f6c89c1c640 1 -- 172.21.15.78:0/4088345220 shutdown_connections 2024-04-25T01:43:23.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.362+0000 7f6c89c1c640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6c700a2b90 0x7f6c700a5050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.362+0000 7f6c89c1c640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c84102230 0x7f6c8419ee30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.362+0000 7f6c89c1c640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c84101870 0x7f6c8419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.362+0000 7f6c89c1c640 1 --2- 172.21.15.78:0/4088345220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c84100680 0x7f6c84069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.362+0000 7f6c89c1c640 1 -- 172.21.15.78:0/4088345220 >> 172.21.15.78:0/4088345220 conn(0x7f6c840fbbf0 msgr2=0x7f6c84104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:23.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.362+0000 7f6c89c1c640 1 -- 172.21.15.78:0/4088345220 shutdown_connections 2024-04-25T01:43:23.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.362+0000 7f6c89c1c640 1 -- 172.21.15.78:0/4088345220 wait complete. 2024-04-25T01:43:23.377 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:43:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.487+0000 7f004505d640 1 -- 172.21.15.78:0/1110571536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0040100f70 msgr2=0x7f00401013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.487+0000 7f004505d640 1 --2- 172.21.15.78:0/1110571536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0040100f70 0x7f00401013f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0034009900 tx=0x7f003402a410 comp rx=0 tx=0).stop 2024-04-25T01:43:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.487+0000 7f004505d640 1 -- 172.21.15.78:0/1110571536 shutdown_connections 2024-04-25T01:43:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.487+0000 7f004505d640 1 --2- 172.21.15.78:0/1110571536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0040101930 0x7f0040108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.487+0000 7f004505d640 1 --2- 172.21.15.78:0/1110571536 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0040100f70 0x7f00401013f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.487+0000 7f004505d640 1 --2- 172.21.15.78:0/1110571536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00400ffe20 0x7f0040100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.487+0000 7f004505d640 1 -- 172.21.15.78:0/1110571536 >> 172.21.15.78:0/1110571536 conn(0x7f00400fb3b0 msgr2=0x7f00400fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.487+0000 7f004505d640 1 -- 172.21.15.78:0/1110571536 shutdown_connections 2024-04-25T01:43:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.487+0000 7f004505d640 1 -- 172.21.15.78:0/1110571536 wait complete. 2024-04-25T01:43:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f004505d640 1 Processor -- start 2024-04-25T01:43:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f004505d640 1 -- start start 2024-04-25T01:43:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f004505d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00400ffe20 0x7f00401097b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f004505d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0040100f70 0x7f004019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f004505d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0040101930 0x7f004019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f004505d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f004010c350 con 0x7f0040101930 2024-04-25T01:43:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f004505d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f004010c1d0 con 0x7f0040100f70 2024-04-25T01:43:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f004505d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f004010c4d0 con 0x7f00400ffe20 2024-04-25T01:43:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f003f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0040101930 0x7f004019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:23.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f003f59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0040101930 0x7f004019ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38662/0 (socket says 172.21.15.78:38662) 2024-04-25T01:43:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f003f59e640 1 -- 172.21.15.78:0/2127459844 learned_addr learned my addr 172.21.15.78:0/2127459844 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f003ed9d640 1 --2- 172.21.15.78:0/2127459844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00400ffe20 0x7f00401097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f003e59c640 1 --2- 172.21.15.78:0/2127459844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0040100f70 0x7f004019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f003f59e640 1 -- 172.21.15.78:0/2127459844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00400ffe20 msgr2=0x7f00401097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f003f59e640 1 --2- 172.21.15.78:0/2127459844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00400ffe20 0x7f00401097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f003f59e640 1 -- 172.21.15.78:0/2127459844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0040100f70 msgr2=0x7f004019c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f003f59e640 1 --2- 172.21.15.78:0/2127459844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0040100f70 0x7f004019c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f003f59e640 1 -- 172.21.15.78:0/2127459844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004019f750 con 0x7f0040101930 2024-04-25T01:43:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.488+0000 7f003f59e640 1 --2- 172.21.15.78:0/2127459844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0040101930 0x7f004019ee40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f003800cad0 tx=0x7f003800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.489+0000 7f0027fff640 1 -- 172.21.15.78:0/2127459844 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003800ae20 con 0x7f0040101930 2024-04-25T01:43:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.489+0000 7f004505d640 1 -- 172.21.15.78:0/2127459844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004019f9e0 con 0x7f0040101930 2024-04-25T01:43:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.489+0000 7f0027fff640 1 -- 172.21.15.78:0/2127459844 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0038005020 con 0x7f0040101930 2024-04-25T01:43:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.489+0000 7f0027fff640 1 -- 172.21.15.78:0/2127459844 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003800f6f0 con 0x7f0040101930 2024-04-25T01:43:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.489+0000 7f004505d640 1 -- 172.21.15.78:0/2127459844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004019feb0 con 0x7f0040101930 2024-04-25T01:43:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.490+0000 7f004505d640 1 -- 172.21.15.78:0/2127459844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00401071b0 con 0x7f0040101930 2024-04-25T01:43:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.491+0000 7f0027fff640 1 -- 172.21.15.78:0/2127459844 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f00380051d0 con 0x7f0040101930 2024-04-25T01:43:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.491+0000 7f0027fff640 1 --2- 172.21.15.78:0/2127459844 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f00280a2b90 0x7f00280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.491+0000 7f003ed9d640 1 --2- 172.21.15.78:0/2127459844 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f00280a2b90 0x7f00280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.491+0000 7f0027fff640 1 -- 172.21.15.78:0/2127459844 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f00380cf1f0 con 0x7f0040101930 2024-04-25T01:43:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.492+0000 7f003ed9d640 1 --2- 172.21.15.78:0/2127459844 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f00280a2b90 0x7f00280a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0030005fd0 tx=0x7f003000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:23.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.496+0000 7f0027fff640 1 -- 172.21.15.78:0/2127459844 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0038014030 con 0x7f0040101930 2024-04-25T01:43:23.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.708+0000 7f004505d640 1 -- 172.21.15.78:0/2127459844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0040062340 con 0x7f0040101930 2024-04-25T01:43:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.708+0000 7f0027fff640 1 -- 172.21.15.78:0/2127459844 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11809 (secure 0 0 0) 0x7f003809b960 con 0x7f0040101930 2024-04-25T01:43:23.699 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:23.699 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/1337519666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1337519666},{"type":"v1","addr":"172.21.15.78:6839","nonce":1337519666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1963410683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1963410683},{"type":"v1","addr":"172.21.15.78:6841","nonce":1963410683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24379,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:43:17.355192+0000","modified":"2024-04-25T01:43:21.781823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1969344843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1969344843},{"type":"v1","addr":"172.21.15.152:6839","nonce":1969344843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2024-04-25T01:43:23.699 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 17 2024-04-25T01:43:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.710+0000 7f004505d640 1 -- 172.21.15.78:0/2127459844 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f00280a2b90 msgr2=0x7f00280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.710+0000 7f004505d640 1 --2- 172.21.15.78:0/2127459844 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f00280a2b90 0x7f00280a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0030005fd0 tx=0x7f003000b390 comp rx=0 tx=0).stop 2024-04-25T01:43:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.710+0000 7f004505d640 1 -- 172.21.15.78:0/2127459844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0040101930 msgr2=0x7f004019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.710+0000 7f004505d640 1 --2- 172.21.15.78:0/2127459844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0040101930 0x7f004019ee40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f003800cad0 tx=0x7f003800cf90 comp rx=0 tx=0).stop 2024-04-25T01:43:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.710+0000 7f004505d640 1 -- 172.21.15.78:0/2127459844 shutdown_connections 2024-04-25T01:43:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.710+0000 7f004505d640 1 --2- 172.21.15.78:0/2127459844 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f00280a2b90 0x7f00280a5050 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.710+0000 7f004505d640 1 --2- 172.21.15.78:0/2127459844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0040101930 0x7f004019ee40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.710+0000 7f004505d640 1 --2- 172.21.15.78:0/2127459844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0040100f70 0x7f004019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.710+0000 7f004505d640 1 --2- 172.21.15.78:0/2127459844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00400ffe20 0x7f00401097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.710+0000 7f004505d640 1 -- 172.21.15.78:0/2127459844 >> 172.21.15.78:0/2127459844 conn(0x7f00400fb3b0 msgr2=0x7f0040104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.711+0000 7f004505d640 1 -- 172.21.15.78:0/2127459844 shutdown_connections 2024-04-25T01:43:23.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.711+0000 7f004505d640 1 -- 172.21.15.78:0/2127459844 wait complete. 2024-04-25T01:43:23.711 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:23 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/4060094538' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-25T01:43:23.711 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:23 smithi078 ceph-mon[29469]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T01:43:23.711 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:23 smithi078 ceph-mon[29469]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:23.711 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:23 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/351739492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:23.711 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:23 smithi078 ceph-mon[29469]: pgmap v293: 129 pgs: 101 active+clean, 3 creating+peering, 25 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-04-25T01:43:23.711 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:23 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/664122544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:23.711 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:23 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/4088345220' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:23.713 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:43:23.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:23 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/4060094538' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-25T01:43:23.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:23 smithi152 ceph-mon[32465]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T01:43:23.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:23 smithi152 ceph-mon[32465]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:23.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:23 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/351739492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:23.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:23 smithi152 ceph-mon[32465]: pgmap v293: 129 pgs: 101 active+clean, 3 creating+peering, 25 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-04-25T01:43:23.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:23 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/664122544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:23.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:23 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/4088345220' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:23.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.819+0000 7fc2ec516640 1 -- 172.21.15.78:0/1375215297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2e4103350 msgr2=0x7fc2e4109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.819+0000 7fc2ec516640 1 --2- 172.21.15.78:0/1375215297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2e4103350 0x7fc2e4109be0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009ca0 tx=0x7fc2dc02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.819+0000 7fc2ec516640 1 -- 172.21.15.78:0/1375215297 shutdown_connections 2024-04-25T01:43:23.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.819+0000 7fc2ec516640 1 --2- 172.21.15.78:0/1375215297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2e4103350 0x7fc2e4109be0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.819+0000 7fc2ec516640 1 --2- 172.21.15.78:0/1375215297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2e4102990 0x7fc2e4102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.819+0000 7fc2ec516640 1 --2- 172.21.15.78:0/1375215297 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2e41017a0 0x7fc2e4101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.819+0000 7fc2ec516640 1 -- 172.21.15.78:0/1375215297 >> 172.21.15.78:0/1375215297 conn(0x7fc2e40fccf0 msgr2=0x7fc2e40ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:23.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.819+0000 7fc2ec516640 1 -- 172.21.15.78:0/1375215297 shutdown_connections 2024-04-25T01:43:23.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.819+0000 7fc2ec516640 1 -- 172.21.15.78:0/1375215297 wait complete. 2024-04-25T01:43:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.820+0000 7fc2ec516640 1 Processor -- start 2024-04-25T01:43:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.820+0000 7fc2ec516640 1 -- start start 2024-04-25T01:43:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.820+0000 7fc2ec516640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2e41017a0 0x7fc2e4197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.820+0000 7fc2ec516640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2e4102990 0x7fc2e4197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.820+0000 7fc2ec516640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2e4103350 0x7fc2e419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.820+0000 7fc2ec516640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e410d400 con 0x7fc2e4103350 2024-04-25T01:43:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.820+0000 7fc2ec516640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e410d280 con 0x7fc2e4102990 2024-04-25T01:43:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.820+0000 7fc2ec516640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e410d580 con 0x7fc2e41017a0 2024-04-25T01:43:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.820+0000 7fc2ea2b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2e41017a0 0x7fc2e4197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.820+0000 7fc2eaab3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2e4103350 0x7fc2e419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.820+0000 7fc2ea2b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2e41017a0 0x7fc2e4197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.78:50200/0 (socket says 172.21.15.78:50200) 2024-04-25T01:43:23.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.820+0000 7fc2ea2b2640 1 -- 172.21.15.78:0/4292280379 learned_addr learned my addr 172.21.15.78:0/4292280379 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.820+0000 7fc2e9ab1640 1 --2- 172.21.15.78:0/4292280379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2e4102990 0x7fc2e4197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.821+0000 7fc2eaab3640 1 -- 172.21.15.78:0/4292280379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2e41017a0 msgr2=0x7fc2e4197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.821+0000 7fc2eaab3640 1 --2- 172.21.15.78:0/4292280379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2e41017a0 0x7fc2e4197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.821+0000 7fc2eaab3640 1 -- 172.21.15.78:0/4292280379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2e4102990 msgr2=0x7fc2e4197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.821+0000 7fc2eaab3640 1 --2- 172.21.15.78:0/4292280379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2e4102990 0x7fc2e4197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.821+0000 7fc2eaab3640 1 -- 172.21.15.78:0/4292280379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e419f660 con 0x7fc2e4103350 2024-04-25T01:43:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.821+0000 7fc2eaab3640 1 --2- 172.21.15.78:0/4292280379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2e4103350 0x7fc2e419eff0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc031ae0 tx=0x7fc2dc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.821+0000 7fc2db7fe640 1 -- 172.21.15.78:0/4292280379 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc003c30 con 0x7fc2e4103350 2024-04-25T01:43:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.821+0000 7fc2db7fe640 1 -- 172.21.15.78:0/4292280379 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2dc003dd0 con 0x7fc2e4103350 2024-04-25T01:43:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.821+0000 7fc2ec516640 1 -- 172.21.15.78:0/4292280379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e4073970 con 0x7fc2e4103350 2024-04-25T01:43:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.821+0000 7fc2db7fe640 1 -- 172.21.15.78:0/4292280379 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc02fcc0 con 0x7fc2e4103350 2024-04-25T01:43:23.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.821+0000 7fc2ec516640 1 -- 172.21.15.78:0/4292280379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e4073e90 con 0x7fc2e4103350 2024-04-25T01:43:23.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.822+0000 7fc2ec516640 1 -- 172.21.15.78:0/4292280379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e4108a60 con 0x7fc2e4103350 2024-04-25T01:43:23.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.823+0000 7fc2db7fe640 1 -- 172.21.15.78:0/4292280379 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc2dc004050 con 0x7fc2e4103350 2024-04-25T01:43:23.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.824+0000 7fc2db7fe640 1 --2- 172.21.15.78:0/4292280379 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc2d00a2b90 0x7fc2d00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:23.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.824+0000 7fc2ea2b2640 1 --2- 172.21.15.78:0/4292280379 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc2d00a2b90 0x7fc2d00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:23.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.824+0000 7fc2db7fe640 1 -- 172.21.15.78:0/4292280379 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc2dc0f02f0 con 0x7fc2e4103350 2024-04-25T01:43:23.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.824+0000 7fc2ea2b2640 1 --2- 172.21.15.78:0/4292280379 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc2d00a2b90 0x7fc2d00a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000bc80 tx=0x7fc2e000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:23.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:23.828+0000 7fc2db7fe640 1 -- 172.21.15.78:0/4292280379 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc2dc0bd010 con 0x7fc2e4103350 2024-04-25T01:43:23.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:23 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/4060094538' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-25T01:43:23.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:23 smithi196 ceph-mon[32978]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T01:43:23.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:23 smithi196 ceph-mon[32978]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:23.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:23 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/351739492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:23.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:23 smithi196 ceph-mon[32978]: pgmap v293: 129 pgs: 101 active+clean, 3 creating+peering, 25 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-04-25T01:43:23.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:23 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/664122544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:23.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:23 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/4088345220' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:23.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.005+0000 7fc2ec516640 1 -- 172.21.15.78:0/4292280379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2e4075270 con 0x7fc2e4103350 2024-04-25T01:43:23.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.007+0000 7fc2db7fe640 1 -- 172.21.15.78:0/4292280379 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24151 (secure 0 0 0) 0x7fc2dc0bc910 con 0x7fc2e4103350 2024-04-25T01:43:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:23.998 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:43:21.781660+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:43:23.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.009+0000 7fc2ec516640 1 -- 172.21.15.78:0/4292280379 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc2d00a2b90 msgr2=0x7fc2d00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:23.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.009+0000 7fc2ec516640 1 --2- 172.21.15.78:0/4292280379 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc2d00a2b90 0x7fc2d00a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000bc80 tx=0x7fc2e000b410 comp rx=0 tx=0).stop 2024-04-25T01:43:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.009+0000 7fc2ec516640 1 -- 172.21.15.78:0/4292280379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2e4103350 msgr2=0x7fc2e419eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.009+0000 7fc2ec516640 1 --2- 172.21.15.78:0/4292280379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2e4103350 0x7fc2e419eff0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc031ae0 tx=0x7fc2dc031ee0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.009+0000 7fc2ec516640 1 -- 172.21.15.78:0/4292280379 shutdown_connections 2024-04-25T01:43:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.009+0000 7fc2ec516640 1 --2- 172.21.15.78:0/4292280379 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc2d00a2b90 0x7fc2d00a5050 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.009+0000 7fc2ec516640 1 --2- 172.21.15.78:0/4292280379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2e4103350 0x7fc2e419eff0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.009+0000 7fc2ec516640 1 --2- 172.21.15.78:0/4292280379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2e4102990 0x7fc2e4197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.009+0000 7fc2ec516640 1 --2- 172.21.15.78:0/4292280379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2e41017a0 0x7fc2e4197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.009+0000 7fc2ec516640 1 -- 172.21.15.78:0/4292280379 >> 172.21.15.78:0/4292280379 conn(0x7fc2e40fccf0 msgr2=0x7fc2e4105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.009+0000 7fc2ec516640 1 -- 172.21.15.78:0/4292280379 shutdown_connections 2024-04-25T01:43:24.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.009+0000 7fc2ec516640 1 -- 172.21.15.78:0/4292280379 wait complete. 2024-04-25T01:43:24.011 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T01:43:17.355192+0000', 'modified': '2024-04-25T01:43:21.781823+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24491}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24491': {'gid': 24491, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6839/1969344843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 1969344843}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 1969344843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24491, 'qdb_cluster': [24491]} 2024-04-25T01:43:24.011 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T01:43:24.011 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T01:43:24.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.119+0000 7f2bfbc1c640 1 -- 172.21.15.78:0/2220031168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bf41017a0 msgr2=0x7f2bf4101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:24.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.119+0000 7f2bfbc1c640 1 --2- 172.21.15.78:0/2220031168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bf41017a0 0x7f2bf4101bc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009ca0 tx=0x7f2bf002a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.119+0000 7f2bfbc1c640 1 -- 172.21.15.78:0/2220031168 shutdown_connections 2024-04-25T01:43:24.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.119+0000 7f2bfbc1c640 1 --2- 172.21.15.78:0/2220031168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bf4103350 0x7f2bf4109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.119+0000 7f2bfbc1c640 1 --2- 172.21.15.78:0/2220031168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf4102990 0x7f2bf4102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.119+0000 7f2bfbc1c640 1 --2- 172.21.15.78:0/2220031168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bf41017a0 0x7f2bf4101bc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.119+0000 7f2bfbc1c640 1 -- 172.21.15.78:0/2220031168 >> 172.21.15.78:0/2220031168 conn(0x7f2bf40fcd10 msgr2=0x7f2bf40ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.119+0000 7f2bfbc1c640 1 -- 172.21.15.78:0/2220031168 shutdown_connections 2024-04-25T01:43:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.119+0000 7f2bfbc1c640 1 -- 172.21.15.78:0/2220031168 wait complete. 2024-04-25T01:43:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bfbc1c640 1 Processor -- start 2024-04-25T01:43:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bfbc1c640 1 -- start start 2024-04-25T01:43:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bfbc1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bf41017a0 0x7f2bf4197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bfbc1c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bf4102990 0x7f2bf4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bfbc1c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf4103350 0x7f2bf419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bfbc1c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf410d490 con 0x7f2bf41017a0 2024-04-25T01:43:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bfbc1c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf410d310 con 0x7f2bf4102990 2024-04-25T01:43:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bfbc1c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf410d610 con 0x7f2bf4103350 2024-04-25T01:43:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bf99b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bf41017a0 0x7f2bf4197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bf99b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bf41017a0 0x7f2bf4197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38698/0 (socket says 172.21.15.78:38698) 2024-04-25T01:43:24.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bf99b8640 1 -- 172.21.15.78:0/1005978617 learned_addr learned my addr 172.21.15.78:0/1005978617 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bfa1b9640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf4103350 0x7f2bf419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bf99b8640 1 -- 172.21.15.78:0/1005978617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf4103350 msgr2=0x7f2bf419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bf91b7640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bf4102990 0x7f2bf4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bf99b8640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf4103350 0x7f2bf419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bf99b8640 1 -- 172.21.15.78:0/1005978617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bf4102990 msgr2=0x7f2bf4198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bf99b8640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bf4102990 0x7f2bf4198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bf99b8640 1 -- 172.21.15.78:0/1005978617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf419f770 con 0x7f2bf41017a0 2024-04-25T01:43:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bfa1b9640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf4103350 0x7f2bf419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:43:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bf91b7640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bf4102990 0x7f2bf4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:43:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.120+0000 7f2bf99b8640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bf41017a0 0x7f2bf4197b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009430 tx=0x7f2bf002f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.121+0000 7f2beaffd640 1 -- 172.21.15.78:0/1005978617 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf002fe40 con 0x7f2bf41017a0 2024-04-25T01:43:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.121+0000 7f2beaffd640 1 -- 172.21.15.78:0/1005978617 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf0030aa0 con 0x7f2bf41017a0 2024-04-25T01:43:24.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.121+0000 7f2bfbc1c640 1 -- 172.21.15.78:0/1005978617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf419fa00 con 0x7f2bf41017a0 2024-04-25T01:43:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.121+0000 7f2beaffd640 1 -- 172.21.15.78:0/1005978617 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0046ab0 con 0x7f2bf41017a0 2024-04-25T01:43:24.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.121+0000 7f2bfbc1c640 1 -- 172.21.15.78:0/1005978617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf419ffa0 con 0x7f2bf41017a0 2024-04-25T01:43:24.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.123+0000 7f2beaffd640 1 -- 172.21.15.78:0/1005978617 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2bf0030c50 con 0x7f2bf41017a0 2024-04-25T01:43:24.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.123+0000 7f2bfea0a740 1 Processor -- start 2024-04-25T01:43:24.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.123+0000 7f2bfea0a740 1 -- start start 2024-04-25T01:43:24.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.123+0000 7f2beaffd640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2be00a2c10 0x7f2be00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.123+0000 7f2bfea0a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efac8440 0x5572efac8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.123+0000 7f2bfea0a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efacd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bfea0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efacdfa0 0x5572efad0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bfea0a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572efa2bf00 con 0x5572efac8440 2024-04-25T01:43:24.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bfea0a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5572ef9d4750 con 0x5572efac8e30 2024-04-25T01:43:24.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bfea0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572efa27ec0 con 0x5572efacdfa0 2024-04-25T01:43:24.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bf99b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efac8440 0x5572efac8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bfa1b9640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2be00a2c10 0x7f2be00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bf99b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efac8440 0x5572efac8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38700/0 (socket says 172.21.15.78:38700) 2024-04-25T01:43:24.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bf99b8640 1 -- 172.21.15.78:0/781477409 learned_addr learned my addr 172.21.15.78:0/781477409 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:24.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2beaffd640 1 -- 172.21.15.78:0/1005978617 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2bf00f0970 con 0x7f2bf41017a0 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bf91b7640 1 --2- 172.21.15.78:0/781477409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efacd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bfa1b9640 1 --2- 172.21.15.78:0/781477409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efacdfa0 0x5572efad0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bf99b8640 1 -- 172.21.15.78:0/781477409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efacdfa0 msgr2=0x5572efad0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bf99b8640 1 --2- 172.21.15.78:0/781477409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efacdfa0 0x5572efad0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bf99b8640 1 -- 172.21.15.78:0/781477409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 msgr2=0x5572efacd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bf99b8640 1 --2- 172.21.15.78:0/781477409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efacd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bf99b8640 1 -- 172.21.15.78:0/781477409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572ef7402d0 con 0x5572efac8440 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bf91b7640 1 --2- 172.21.15.78:0/781477409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efacd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bfa1b9640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2be00a2c10 0x7f2be00a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2bec006fd0 tx=0x7f2bec013650 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bf99b8640 1 --2- 172.21.15.78:0/781477409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efac8440 0x5572efac8860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00c56d0 tx=0x7f2bf0030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a3420b0dfd5f02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bdffff640 1 -- 172.21.15.78:0/781477409 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00d6370 con 0x5572efac8440 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bdffff640 1 -- 172.21.15.78:0/781477409 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf0036070 con 0x5572efac8440 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.124+0000 7f2bdffff640 1 -- 172.21.15.78:0/781477409 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00d6190 con 0x5572efac8440 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.125+0000 7f2bfea0a740 1 -- 172.21.15.78:0/781477409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efac8440 msgr2=0x5572efac8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.125+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/781477409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efac8440 0x5572efac8860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00c56d0 tx=0x7f2bf0030fc0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.125+0000 7f2bfea0a740 1 -- 172.21.15.78:0/781477409 shutdown_connections 2024-04-25T01:43:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.125+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/781477409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efacdfa0 0x5572efad0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.125+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/781477409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efacd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.125+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/781477409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efac8440 0x5572efac8860 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.125+0000 7f2bfea0a740 1 -- 172.21.15.78:0/781477409 >> 172.21.15.78:0/781477409 conn(0x5572efac3ab0 msgr2=0x5572efac3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.125+0000 7f2bfea0a740 1 -- 172.21.15.78:0/781477409 shutdown_connections 2024-04-25T01:43:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.125+0000 7f2bfea0a740 1 -- 172.21.15.78:0/781477409 wait complete. 2024-04-25T01:43:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.125+0000 7f2bfea0a740 1 -- start start 2024-04-25T01:43:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.125+0000 7f2bfea0a740 1 Processor -- start 2024-04-25T01:43:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfea0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efac8440 0x5572efad2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfea0a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efad2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfea0a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfea0a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572efa2bf00 con 0x5572efacdfa0 2024-04-25T01:43:24.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfea0a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5572ef9e7cf0 con 0x5572efac8e30 2024-04-25T01:43:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfea0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572efad0d10 con 0x5572efac8440 2024-04-25T01:43:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfea0a740 10 client.0 _ll_register_callbacks cb 0x5572ef9e76e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:43:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfa1b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfa1b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38708/0 (socket says 172.21.15.78:38708) 2024-04-25T01:43:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfa1b9640 1 -- 172.21.15.78:0/1797680583 learned_addr learned my addr 172.21.15.78:0/1797680583 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bf91b7640 1 --2- 172.21.15.78:0/1797680583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efad2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bf99b8640 1 --2- 172.21.15.78:0/1797680583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efac8440 0x5572efad2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfa1b9640 1 -- 172.21.15.78:0/1797680583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efac8440 msgr2=0x5572efad2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:43:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfa1b9640 1 --2- 172.21.15.78:0/1797680583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efac8440 0x5572efad2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfa1b9640 1 -- 172.21.15.78:0/1797680583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 msgr2=0x5572efad2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:43:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfa1b9640 1 --2- 172.21.15.78:0/1797680583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efad2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfa1b9640 1 -- 172.21.15.78:0/1797680583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572ef80a6b0 con 0x5572efacdfa0 2024-04-25T01:43:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bf91b7640 1 --2- 172.21.15.78:0/1797680583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efad2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bfa1b9640 1 --2- 172.21.15.78:0/1797680583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad99c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2bec004690 tx=0x7f2bec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5bf63235931e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:43:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.126+0000 7f2bdd7fa640 1 -- 172.21.15.78:0/1797680583 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec027060 con 0x5572efacdfa0 2024-04-25T01:43:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.127+0000 7f2bdd7fa640 1 -- 172.21.15.78:0/1797680583 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2bec00ae10 con 0x5572efacdfa0 2024-04-25T01:43:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.127+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1797680583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bf0036070 con 0x5572efacdfa0 2024-04-25T01:43:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.127+0000 7f2bdd7fa640 1 -- 172.21.15.78:0/1797680583 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2bec02e050 con 0x5572efacdfa0 2024-04-25T01:43:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.127+0000 7f2bfea0a740 10 client.14772 fetch_fsmap learned FSMap version 17 2024-04-25T01:43:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.127+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1797680583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5572efadca10 con 0x5572efacdfa0 2024-04-25T01:43:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.127+0000 7f2bdd7fa640 1 -- 172.21.15.78:0/1797680583 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf0036070 con 0x5572efacdfa0 2024-04-25T01:43:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.127+0000 7f2bdd7fa640 1 -- 172.21.15.78:0/1797680583 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f2bec015040 con 0x5572efacdfa0 2024-04-25T01:43:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.127+0000 7f2bfea0a740 10 client.14772 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T01:43:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.127+0000 7f2bfea0a740 10 client.14772 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T01:43:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.127+0000 7f2bfea0a740 20 client.14772 populate_metadata read hostname 'smithi078' 2024-04-25T01:43:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.127+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/1797680583 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5572efadf180 0x5572efae05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.127+0000 7f2bfea0a740 4 client.14772 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:43:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.127+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1797680583 --> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572efa39440 con 0x5572efadf180 2024-04-25T01:43:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.127+0000 7f2bf99b8640 1 --2- 172.21.15.78:0/1797680583 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5572efadf180 0x5572efae05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.128+0000 7f2bdd7fa640 10 client.14772 ms_handle_connect on v2:172.21.15.152:6838/1969344843 2024-04-25T01:43:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.128+0000 7f2bf99b8640 1 --2- 172.21.15.78:0/1797680583 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5572efadf180 0x5572efae05f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d8331e0aa73f48f server_cookie=ac6754d5364a429a in_seq=0 out_seq=0 2024-04-25T01:43:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.128+0000 7f2bdd7fa640 1 -- 172.21.15.78:0/1797680583 <== mds.0 v2:172.21.15.152:6838/1969344843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572efa39440 con 0x5572efadf180 2024-04-25T01:43:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.128+0000 7f2bdd7fa640 10 client.14772 handle_command_reply: tid=0 2024-04-25T01:43:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.128+0000 7f2bfea0a740 1 client.14772 shutdown 2024-04-25T01:43:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.129+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1797680583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 msgr2=0x5572efad99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.129+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/1797680583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad99c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2bec004690 tx=0x7f2bec0046c0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.129+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1797680583 shutdown_connections 2024-04-25T01:43:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.129+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/1797680583 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5572efadf180 0x5572efae05f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.129+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/1797680583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad99c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.129+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/1797680583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efad2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.129+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/1797680583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efac8440 0x5572efad2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.129+0000 7f2bdd7fa640 0 client.14772 ms_handle_reset on v2:172.21.15.152:6838/1969344843 2024-04-25T01:43:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.129+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1797680583 >> 172.21.15.78:0/1797680583 conn(0x5572efac3ab0 msgr2=0x5572efac2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.129+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1797680583 shutdown_connections 2024-04-25T01:43:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.129+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1797680583 wait complete. 2024-04-25T01:43:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.129+0000 7f2bfea0a740 20 client.14772 trim_cache size 0 max 16384 2024-04-25T01:43:24.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfea0a740 1 Processor -- start 2024-04-25T01:43:24.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfea0a740 1 -- start start 2024-04-25T01:43:24.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfea0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efac8440 0x5572efad1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfea0a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efad9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfea0a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfea0a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572efade930 con 0x5572efacdfa0 2024-04-25T01:43:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfea0a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5572efadf000 con 0x5572efac8e30 2024-04-25T01:43:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfea0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572efadf6d0 con 0x5572efac8440 2024-04-25T01:43:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfa1b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfa1b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38716/0 (socket says 172.21.15.78:38716) 2024-04-25T01:43:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfa1b9640 1 -- 172.21.15.78:0/3313557956 learned_addr learned my addr 172.21.15.78:0/3313557956 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bf99b8640 1 --2- 172.21.15.78:0/3313557956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efac8440 0x5572efad1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfa1b9640 1 -- 172.21.15.78:0/3313557956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efac8440 msgr2=0x5572efad1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:43:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfa1b9640 1 --2- 172.21.15.78:0/3313557956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efac8440 0x5572efad1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfa1b9640 1 -- 172.21.15.78:0/3313557956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 msgr2=0x5572efad9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:43:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfa1b9640 1 --2- 172.21.15.78:0/3313557956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efad9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfa1b9640 1 -- 172.21.15.78:0/3313557956 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572efac96c0 con 0x5572efacdfa0 2024-04-25T01:43:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bfa1b9640 1 --2- 172.21.15.78:0/3313557956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad1c50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2bec008010 tx=0x7f2bec00a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5eea6445614372c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bdf7fe640 1 -- 172.21.15.78:0/3313557956 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec027040 con 0x5572efacdfa0 2024-04-25T01:43:24.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bdf7fe640 1 -- 172.21.15.78:0/3313557956 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bec00ae10 con 0x5572efacdfa0 2024-04-25T01:43:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.147+0000 7f2bdf7fe640 1 -- 172.21.15.78:0/3313557956 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec00f040 con 0x5572efacdfa0 2024-04-25T01:43:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.148+0000 7f2bfea0a740 1 -- 172.21.15.78:0/3313557956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 msgr2=0x5572efad1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.148+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/3313557956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad1c50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2bec008010 tx=0x7f2bec00a9e0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.148+0000 7f2bfea0a740 1 -- 172.21.15.78:0/3313557956 shutdown_connections 2024-04-25T01:43:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.148+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/3313557956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad1c50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.148+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/3313557956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efad9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.148+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/3313557956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efac8440 0x5572efad1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.148+0000 7f2bfea0a740 1 -- 172.21.15.78:0/3313557956 >> 172.21.15.78:0/3313557956 conn(0x5572efac3ab0 msgr2=0x5572efac3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.148+0000 7f2bfea0a740 1 -- 172.21.15.78:0/3313557956 shutdown_connections 2024-04-25T01:43:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.148+0000 7f2bfea0a740 1 -- 172.21.15.78:0/3313557956 wait complete. 2024-04-25T01:43:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.148+0000 7f2bfea0a740 1 -- start start 2024-04-25T01:43:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.148+0000 7f2bfea0a740 1 Processor -- start 2024-04-25T01:43:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfea0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efac8440 0x5572efad5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfea0a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efad60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfea0a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfea0a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5572efae0470 con 0x5572efacdfa0 2024-04-25T01:43:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfea0a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5572efadfda0 con 0x5572efac8e30 2024-04-25T01:43:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfea0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572efae0b40 con 0x5572efac8440 2024-04-25T01:43:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfea0a740 10 client.0 _ll_register_callbacks cb 0x7f2bec018730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:43:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfa1b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfa1b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38732/0 (socket says 172.21.15.78:38732) 2024-04-25T01:43:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfa1b9640 1 -- 172.21.15.78:0/1430349705 learned_addr learned my addr 172.21.15.78:0/1430349705 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfa1b9640 1 -- 172.21.15.78:0/1430349705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efac8440 msgr2=0x5572efad5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:43:24.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfa1b9640 1 --2- 172.21.15.78:0/1430349705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efac8440 0x5572efad5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfa1b9640 1 -- 172.21.15.78:0/1430349705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 msgr2=0x5572efad60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:43:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfa1b9640 1 --2- 172.21.15.78:0/1430349705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efad60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfa1b9640 1 -- 172.21.15.78:0/1430349705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572efad94f0 con 0x5572efacdfa0 2024-04-25T01:43:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfa1b9640 1 --2- 172.21.15.78:0/1430349705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad8f30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2bec008080 tx=0x7f2bec022720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924d118c3a4459ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:43:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bdd7fa640 1 -- 172.21.15.78:0/1430349705 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec042060 con 0x5572efacdfa0 2024-04-25T01:43:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bdd7fa640 1 -- 172.21.15.78:0/1430349705 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2bec005bd0 con 0x5572efacdfa0 2024-04-25T01:43:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1430349705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bec00ae10 con 0x5572efacdfa0 2024-04-25T01:43:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bdd7fa640 1 -- 172.21.15.78:0/1430349705 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2bec049050 con 0x5572efacdfa0 2024-04-25T01:43:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfea0a740 10 client.14778 fetch_fsmap learned FSMap version 17 2024-04-25T01:43:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.149+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1430349705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5572efb45a70 con 0x5572efacdfa0 2024-04-25T01:43:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.150+0000 7f2bdd7fa640 1 -- 172.21.15.78:0/1430349705 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bec00ae10 con 0x5572efacdfa0 2024-04-25T01:43:24.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.150+0000 7f2bdd7fa640 1 -- 172.21.15.78:0/1430349705 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f2bec015040 con 0x5572efacdfa0 2024-04-25T01:43:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.150+0000 7f2bfea0a740 10 client.14778 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T01:43:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.150+0000 7f2bfea0a740 10 client.14778 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T01:43:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.150+0000 7f2bfea0a740 20 client.14778 populate_metadata read hostname 'smithi078' 2024-04-25T01:43:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.150+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/1430349705 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5572efb4a2d0 0x5572efb48fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.150+0000 7f2bfea0a740 4 client.14778 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:43:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.150+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1430349705 --> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] -- command(tid 0: {"prefix": "status"}) -- 0x5572efa33790 con 0x5572efb4a2d0 2024-04-25T01:43:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.150+0000 7f2bf99b8640 1 --2- 172.21.15.78:0/1430349705 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5572efb4a2d0 0x5572efb48fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bdd7fa640 10 client.14778 ms_handle_connect on v2:172.21.15.152:6838/1969344843 2024-04-25T01:43:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bf99b8640 1 --2- 172.21.15.78:0/1430349705 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5572efb4a2d0 0x5572efb48fe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9a762a651e189d9 server_cookie=19fc9c1e46e918a0 in_seq=0 out_seq=0 2024-04-25T01:43:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bdd7fa640 1 -- 172.21.15.78:0/1430349705 <== mds.0 v2:172.21.15.152:6838/1969344843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5572efa33790 con 0x5572efb4a2d0 2024-04-25T01:43:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bdd7fa640 10 client.14778 handle_command_reply: tid=0 2024-04-25T01:43:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bfea0a740 1 client.14778 shutdown 2024-04-25T01:43:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1430349705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 msgr2=0x5572efad8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:24.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/1430349705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad8f30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2bec008080 tx=0x7f2bec022720 comp rx=0 tx=0).stop 2024-04-25T01:43:24.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1430349705 shutdown_connections 2024-04-25T01:43:24.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/1430349705 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5572efb4a2d0 0x5572efb48fe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/1430349705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5572efacdfa0 0x5572efad8f30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/1430349705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5572efac8e30 0x5572efad60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bdd7fa640 0 client.14778 ms_handle_reset on v2:172.21.15.152:6838/1969344843 2024-04-25T01:43:24.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bfea0a740 1 --2- 172.21.15.78:0/1430349705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efac8440 0x5572efad5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1430349705 >> 172.21.15.78:0/1430349705 conn(0x5572efac3ab0 msgr2=0x5572efad1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:24.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1430349705 shutdown_connections 2024-04-25T01:43:24.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bfea0a740 1 -- 172.21.15.78:0/1430349705 wait complete. 2024-04-25T01:43:24.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.151+0000 7f2bfea0a740 20 client.14778 trim_cache size 0 max 16384 2024-04-25T01:43:24.150 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:43:24.150 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "df36bea8-02a3-11ef-bc93-c7b262605968", 2024-04-25T01:43:24.150 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-04-25T01:43:24.150 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24491, 2024-04-25T01:43:24.150 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T01:43:24.150 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T01:43:24.150 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T01:43:24.150 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 6.1244671159999999, 2024-04-25T01:43:24.150 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 17, 2024-04-25T01:43:24.150 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 90, 2024-04-25T01:43:24.150 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T01:43:24.150 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 12.219352271 2024-04-25T01:43:24.151 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:43:24.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.152+0000 7f2bfbc1c640 1 -- 172.21.15.78:0/1005978617 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2be00a2c10 msgr2=0x7f2be00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:24.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.152+0000 7f2bfbc1c640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2be00a2c10 0x7f2be00a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2bec006fd0 tx=0x7f2bec013650 comp rx=0 tx=0).stop 2024-04-25T01:43:24.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.152+0000 7f2bfbc1c640 1 -- 172.21.15.78:0/1005978617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bf41017a0 msgr2=0x7f2bf4197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:24.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.152+0000 7f2bfbc1c640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bf41017a0 0x7f2bf4197b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009430 tx=0x7f2bf002f840 comp rx=0 tx=0).stop 2024-04-25T01:43:24.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.152+0000 7f2bfbc1c640 1 -- 172.21.15.78:0/1005978617 shutdown_connections 2024-04-25T01:43:24.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.152+0000 7f2bfbc1c640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf4103350 0x7f2bf419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.152+0000 7f2bfbc1c640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2bf4102990 0x7f2bf4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.152+0000 7f2bfbc1c640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2be00a2c10 0x7f2be00a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.152+0000 7f2bfbc1c640 1 --2- 172.21.15.78:0/1005978617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2bf41017a0 0x7f2bf4197b50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:24.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.152+0000 7f2bfbc1c640 1 -- 172.21.15.78:0/1005978617 >> 172.21.15.78:0/1005978617 conn(0x7f2bf40fcd10 msgr2=0x7f2bf4105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:24.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.152+0000 7f2bfbc1c640 1 -- 172.21.15.78:0/1005978617 shutdown_connections 2024-04-25T01:43:24.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:24.152+0000 7f2bfbc1c640 1 -- 172.21.15.78:0/1005978617 wait complete. 2024-04-25T01:43:24.153 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:43:24.154 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:43:24.154 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T01:43:24.388 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:43:24.755 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:24 smithi078 ceph-mon[29469]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T01:43:24.755 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:24 smithi078 ceph-mon[29469]: Cluster is now healthy 2024-04-25T01:43:24.755 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:24 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2127459844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:24.755 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:24 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/4292280379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:24.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:24 smithi152 ceph-mon[32465]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T01:43:24.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:24 smithi152 ceph-mon[32465]: Cluster is now healthy 2024-04-25T01:43:24.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:24 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2127459844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:24.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:24 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/4292280379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:24.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:24 smithi196 ceph-mon[32978]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T01:43:24.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:24 smithi196 ceph-mon[32978]: Cluster is now healthy 2024-04-25T01:43:24.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:24 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2127459844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:24.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:24 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/4292280379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:25.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:25 smithi078 ceph-mon[29469]: pgmap v294: 129 pgs: 125 active+clean, 3 creating+peering, 1 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 934 B/s wr, 5 op/s 2024-04-25T01:43:25.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:25 smithi152 ceph-mon[32465]: pgmap v294: 129 pgs: 125 active+clean, 3 creating+peering, 1 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 934 B/s wr, 5 op/s 2024-04-25T01:43:25.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:25 smithi196 ceph-mon[32978]: pgmap v294: 129 pgs: 125 active+clean, 3 creating+peering, 1 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 934 B/s wr, 5 op/s 2024-04-25T01:43:26.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.058+0000 7f57b7a22700 1 -- 172.21.15.78:0/2249280193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b01017e0 msgr2=0x7f57b0106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:26.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.058+0000 7f57b7a22700 1 --2- 172.21.15.78:0/2249280193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b01017e0 0x7f57b0106f20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f57a00099c0 tx=0x7f57a0009cd0 comp rx=0 tx=0).stop 2024-04-25T01:43:26.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 -- 172.21.15.78:0/2249280193 shutdown_connections 2024-04-25T01:43:26.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 --2- 172.21.15.78:0/2249280193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b0107650 0x7f57b010f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:26.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 --2- 172.21.15.78:0/2249280193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b0100e30 0x7f57b0101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:26.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 --2- 172.21.15.78:0/2249280193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b01017e0 0x7f57b0106f20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:26.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 -- 172.21.15.78:0/2249280193 >> 172.21.15.78:0/2249280193 conn(0x7f57b0076450 msgr2=0x7f57b0076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:26.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 -- 172.21.15.78:0/2249280193 shutdown_connections 2024-04-25T01:43:26.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 -- 172.21.15.78:0/2249280193 wait complete. 2024-04-25T01:43:26.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 Processor -- start 2024-04-25T01:43:26.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 -- start start 2024-04-25T01:43:26.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b0100e30 0x7f57b019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:26.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b01017e0 0x7f57b019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:26.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b0107650 0x7f57b0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:26.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f57b0112610 con 0x7f57b0107650 2024-04-25T01:43:26.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f57b0112490 con 0x7f57b0100e30 2024-04-25T01:43:26.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b7a22700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57b0112790 con 0x7f57b01017e0 2024-04-25T01:43:26.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.059+0000 7f57b5fbf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b0107650 0x7f57b0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:26.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57b5fbf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b0107650 0x7f57b0198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38742/0 (socket says 172.21.15.78:38742) 2024-04-25T01:43:26.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57b5fbf700 1 -- 172.21.15.78:0/3542133116 learned_addr learned my addr 172.21.15.78:0/3542133116 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:26.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57b57be700 1 --2- 172.21.15.78:0/3542133116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b0100e30 0x7f57b019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:26.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57b4fbd700 1 --2- 172.21.15.78:0/3542133116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b01017e0 0x7f57b019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:26.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57b5fbf700 1 -- 172.21.15.78:0/3542133116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b01017e0 msgr2=0x7f57b019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:26.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57b5fbf700 1 --2- 172.21.15.78:0/3542133116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b01017e0 0x7f57b019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:26.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57b5fbf700 1 -- 172.21.15.78:0/3542133116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b0100e30 msgr2=0x7f57b019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:26.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57b5fbf700 1 --2- 172.21.15.78:0/3542133116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b0100e30 0x7f57b019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:26.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57b5fbf700 1 -- 172.21.15.78:0/3542133116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b0199010 con 0x7f57b0107650 2024-04-25T01:43:26.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57b5fbf700 1 --2- 172.21.15.78:0/3542133116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b0107650 0x7f57b0198830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f57a800b970 tx=0x7f57a800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:26.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57a67fc700 1 -- 172.21.15.78:0/3542133116 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8019070 con 0x7f57b0107650 2024-04-25T01:43:26.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57a67fc700 1 -- 172.21.15.78:0/3542133116 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57a800ce10 con 0x7f57b0107650 2024-04-25T01:43:26.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57a67fc700 1 -- 172.21.15.78:0/3542133116 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8007370 con 0x7f57b0107650 2024-04-25T01:43:26.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57b7a22700 1 -- 172.21.15.78:0/3542133116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b0199300 con 0x7f57b0107650 2024-04-25T01:43:26.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.060+0000 7f57b7a22700 1 -- 172.21.15.78:0/3542133116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b01a54c0 con 0x7f57b0107650 2024-04-25T01:43:26.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.061+0000 7f57b7a22700 1 -- 172.21.15.78:0/3542133116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b006a480 con 0x7f57b0107650 2024-04-25T01:43:26.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.062+0000 7f57a67fc700 1 -- 172.21.15.78:0/3542133116 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f57a801c070 con 0x7f57b0107650 2024-04-25T01:43:26.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.063+0000 7f57a67fc700 1 --2- 172.21.15.78:0/3542133116 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f579c0a2c20 0x7f579c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:26.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.063+0000 7f57b57be700 1 --2- 172.21.15.78:0/3542133116 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f579c0a2c20 0x7f579c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:26.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.063+0000 7f57a67fc700 1 -- 172.21.15.78:0/3542133116 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f57a80ceb00 con 0x7f57b0107650 2024-04-25T01:43:26.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.063+0000 7f57b57be700 1 --2- 172.21.15.78:0/3542133116 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f579c0a2c20 0x7f579c0a50e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f57ac005950 tx=0x7f57ac00a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:26.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.066+0000 7f57a67fc700 1 -- 172.21.15.78:0/3542133116 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f57a809b820 con 0x7f57b0107650 2024-04-25T01:43:26.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.309+0000 7f57b7a22700 1 -- 172.21.15.78:0/3542133116 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f57b0199d40 con 0x7f57b0107650 2024-04-25T01:43:26.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.310+0000 7f57a67fc700 1 -- 172.21.15.78:0/3542133116 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f57b0199d40 con 0x7f57b0107650 2024-04-25T01:43:26.300 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T01:43:26.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.312+0000 7f57b7a22700 1 -- 172.21.15.78:0/3542133116 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f579c0a2c20 msgr2=0x7f579c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:26.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.312+0000 7f57b7a22700 1 --2- 172.21.15.78:0/3542133116 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f579c0a2c20 0x7f579c0a50e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f57ac005950 tx=0x7f57ac00a300 comp rx=0 tx=0).stop 2024-04-25T01:43:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.312+0000 7f57b7a22700 1 -- 172.21.15.78:0/3542133116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b0107650 msgr2=0x7f57b0198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.312+0000 7f57b7a22700 1 --2- 172.21.15.78:0/3542133116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b0107650 0x7f57b0198830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f57a800b970 tx=0x7f57a800bc80 comp rx=0 tx=0).stop 2024-04-25T01:43:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.312+0000 7f57b7a22700 1 -- 172.21.15.78:0/3542133116 shutdown_connections 2024-04-25T01:43:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.312+0000 7f57b7a22700 1 --2- 172.21.15.78:0/3542133116 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f579c0a2c20 0x7f579c0a50e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.312+0000 7f57b7a22700 1 --2- 172.21.15.78:0/3542133116 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57b0107650 0x7f57b0198830 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.312+0000 7f57b7a22700 1 --2- 172.21.15.78:0/3542133116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b0100e30 0x7f57b019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.312+0000 7f57b7a22700 1 --2- 172.21.15.78:0/3542133116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57b01017e0 0x7f57b019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.312+0000 7f57b7a22700 1 -- 172.21.15.78:0/3542133116 >> 172.21.15.78:0/3542133116 conn(0x7f57b0076450 msgr2=0x7f57b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.312+0000 7f57b7a22700 1 -- 172.21.15.78:0/3542133116 shutdown_connections 2024-04-25T01:43:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:26.312+0000 7f57b7a22700 1 -- 172.21.15.78:0/3542133116 wait complete. 2024-04-25T01:43:26.661 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:26 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3542133116' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T01:43:26.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:26 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3542133116' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T01:43:26.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:26 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3542133116' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T01:43:26.997 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T01:43:26.997 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:43:26.997 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-04-25T01:43:26.997 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T01:43:26.997 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:43:26.997 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T01:43:26.997 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:43:26.997 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:43:26.998 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T01:43:26.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:43:26.998 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-04-25T01:43:27.013 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:43:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:43:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:43:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:43:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-04-25T01:43:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:43:27.014 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:43:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:43:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-04-25T01:43:27.014 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:43:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:43:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-04-25T01:43:27.014 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:43:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T01:43:27.014 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:43:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T01:43:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T01:43:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42174sec preferred_lft 42174sec 2024-04-25T01:43:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::7539:9f76:2904:c153/64 scope link noprefixroute 2024-04-25T01:43:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:43:27.015 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T01:43:27.015 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:43:27.015 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T01:43:27.015 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add name ceph-brx type bridge 2024-04-25T01:43:27.015 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr flush dev ceph-brx 2024-04-25T01:43:27.016 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set ceph-brx up 2024-04-25T01:43:27.016 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T01:43:27.016 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T01:43:27.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:43:27.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:43:27.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:43:27.193 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:43:27.217 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-04-25T01:43:27.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:43:27.218 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-04-25T01:43:27.304 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-04-25T01:43:27.304 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:43:27.305 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T01:43:27.305 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T01:43:27.305 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:43:27.305 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:43:27.305 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T01:43:27.305 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T01:43:27.305 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T01:43:27.305 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T01:43:27.306 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T01:43:27.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:43:27.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:43:27.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:43:27.525 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-04-25T01:43:27.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:43:27.540 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list-id 2024-04-25T01:43:27.594 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:43:27.594 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T01:43:27.594 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:43:27.594 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T01:43:27.594 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T01:43:27.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:43:27.689 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:27 smithi078 ceph-mon[29469]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 632 B/s wr, 1 op/s 2024-04-25T01:43:27.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:43:27.709 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T01:43:27.709 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:43:27.709 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T01:43:27.709 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T01:43:27.710 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T01:43:27.710 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T01:43:27.710 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T01:43:27.710 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T01:43:27.710 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T01:43:27.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:43:27.797 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:27 smithi152 ceph-mon[32465]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 632 B/s wr, 1 op/s 2024-04-25T01:43:27.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:43:27.831 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:43:27.831 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T01:43:27.831 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-04-25T01:43:27.831 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T01:43:27.831 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T01:43:27.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:27 smithi196 ceph-mon[32978]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 632 B/s wr, 1 op/s 2024-04-25T01:43:27.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:43:27.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:43:27.906 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T01:43:27.906 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:43:27.906 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:43:27.921 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T01:43:27.921 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:43:27.921 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:43:27.976 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2024-04-25T01:43:28.042 DEBUG:teuthology.orchestra.run.smithi078:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:43:28.098 INFO:teuthology.orchestra.run.smithi078.stdout:/proc 2024-04-25T01:43:28.098 INFO:teuthology.orchestra.run.smithi078.stdout:/sys 2024-04-25T01:43:28.098 INFO:teuthology.orchestra.run.smithi078.stdout:/dev 2024-04-25T01:43:28.099 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/security 2024-04-25T01:43:28.099 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/shm 2024-04-25T01:43:28.099 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/pts 2024-04-25T01:43:28.099 INFO:teuthology.orchestra.run.smithi078.stdout:/run 2024-04-25T01:43:28.099 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/cgroup 2024-04-25T01:43:28.099 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/pstore 2024-04-25T01:43:28.099 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/bpf 2024-04-25T01:43:28.100 INFO:teuthology.orchestra.run.smithi078.stdout:/ 2024-04-25T01:43:28.100 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/selinux 2024-04-25T01:43:28.100 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:43:28.100 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/mqueue 2024-04-25T01:43:28.100 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/hugepages 2024-04-25T01:43:28.100 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/debug 2024-04-25T01:43:28.101 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/tracing 2024-04-25T01:43:28.101 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:43:28.101 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/config 2024-04-25T01:43:28.101 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/fuse/connections 2024-04-25T01:43:28.101 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T01:43:28.101 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T01:43:28.102 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T01:43:28.102 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T01:43:28.102 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/1000 2024-04-25T01:43:28.103 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/ceph 2024-04-25T01:43:28.103 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay 2024-04-25T01:43:28.104 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/3e54eafd29f33c4eb2c893ab700ef019bffbaf3ddae96b01b5a3fda531eacff9/merged 2024-04-25T01:43:28.104 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/e503707471b636ca9660752ce2439fd2e07680472f022153beabcad30d674f33/merged 2024-04-25T01:43:28.104 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/0 2024-04-25T01:43:28.104 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:43:28.104 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/42a7129716527ade802da9a650f9dee6556b81a9a7c20f10a53f7bc06483cac7/merged 2024-04-25T01:43:28.104 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/0149cce0ee91303fa4f221a12207ccd021407284183f21deadde0a69b3508523/merged 2024-04-25T01:43:28.105 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/989b266395a848cb215aaefbef007e711b82988fbd198c9a7869a9e046a67d54/merged 2024-04-25T01:43:28.105 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/03039d0bd72f9617221e4154c017489596a892a9135d8b4014eae2c3d26553e5/merged 2024-04-25T01:43:28.105 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/c4baff8f9783a66d0b726efaa96dce69f6b978baf4ac5d053d27346c5fee919d/merged 2024-04-25T01:43:28.105 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/ec0bc67b50d348291d7f62767c1ad371c87d899161d1d2d9bafbecb67c8c8022/merged 2024-04-25T01:43:28.105 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/81b2a13efd728c3fdbd5edf227ca55ce95c955bdb119feb433f8352075ea99c1/merged 2024-04-25T01:43:28.105 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/877c9ec6efa063e30073d2e21f7c7d2f20eb0bff231cbf93e9b59eb1965bf611/merged 2024-04-25T01:43:28.106 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns 2024-04-25T01:43:28.106 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:43:28.106 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:43:28.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:43:28.106 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2024-04-25T01:43:28.153 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T01:43:28.154 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T01:43:28.196 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2024-04-25T01:43:28.219 DEBUG:teuthology.orchestra.run.smithi078:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:43:28.274 INFO:teuthology.orchestra.run.smithi078.stdout:/proc 2024-04-25T01:43:28.275 INFO:teuthology.orchestra.run.smithi078.stdout:/sys 2024-04-25T01:43:28.275 INFO:teuthology.orchestra.run.smithi078.stdout:/dev 2024-04-25T01:43:28.275 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/security 2024-04-25T01:43:28.275 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/shm 2024-04-25T01:43:28.275 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/pts 2024-04-25T01:43:28.275 INFO:teuthology.orchestra.run.smithi078.stdout:/run 2024-04-25T01:43:28.275 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/cgroup 2024-04-25T01:43:28.275 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/pstore 2024-04-25T01:43:28.275 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/bpf 2024-04-25T01:43:28.275 INFO:teuthology.orchestra.run.smithi078.stdout:/ 2024-04-25T01:43:28.275 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/selinux 2024-04-25T01:43:28.275 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:43:28.275 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/mqueue 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/hugepages 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/debug 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/tracing 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/config 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/fuse/connections 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/1000 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/ceph 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/3e54eafd29f33c4eb2c893ab700ef019bffbaf3ddae96b01b5a3fda531eacff9/merged 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/e503707471b636ca9660752ce2439fd2e07680472f022153beabcad30d674f33/merged 2024-04-25T01:43:28.276 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/0 2024-04-25T01:43:28.277 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:43:28.277 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/42a7129716527ade802da9a650f9dee6556b81a9a7c20f10a53f7bc06483cac7/merged 2024-04-25T01:43:28.277 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/0149cce0ee91303fa4f221a12207ccd021407284183f21deadde0a69b3508523/merged 2024-04-25T01:43:28.277 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/989b266395a848cb215aaefbef007e711b82988fbd198c9a7869a9e046a67d54/merged 2024-04-25T01:43:28.277 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/03039d0bd72f9617221e4154c017489596a892a9135d8b4014eae2c3d26553e5/merged 2024-04-25T01:43:28.277 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/c4baff8f9783a66d0b726efaa96dce69f6b978baf4ac5d053d27346c5fee919d/merged 2024-04-25T01:43:28.277 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/ec0bc67b50d348291d7f62767c1ad371c87d899161d1d2d9bafbecb67c8c8022/merged 2024-04-25T01:43:28.277 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/81b2a13efd728c3fdbd5edf227ca55ce95c955bdb119feb433f8352075ea99c1/merged 2024-04-25T01:43:28.277 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/877c9ec6efa063e30073d2e21f7c7d2f20eb0bff231cbf93e9b59eb1965bf611/merged 2024-04-25T01:43:28.277 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns 2024-04-25T01:43:28.277 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:43:28.277 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:43:28.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:43:28.278 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2024-04-25T01:43:28.297 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi078.stderr:2024-04-25T01:43:28.306+0000 7f96ad51b480 -1 init, newargv = 0x5557585f20e0 newargc=15 2024-04-25T01:43:28.297 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi078.stderr:ceph-fuse[70628]: starting ceph client 2024-04-25T01:43:28.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi078.stderr:ceph-fuse[70628]: starting fuse 2024-04-25T01:43:28.311 INFO:teuthology.orchestra.run.smithi078.stdout:115 2024-04-25T01:43:28.312 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-25T01:43:28.312 DEBUG:teuthology.orchestra.run.smithi078:> sudo stdin-killer -- python3 -c ' 2024-04-25T01:43:28.312 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2024-04-25T01:43:28.312 DEBUG:teuthology.orchestra.run.smithi078:> import re 2024-04-25T01:43:28.312 DEBUG:teuthology.orchestra.run.smithi078:> import os 2024-04-25T01:43:28.312 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2024-04-25T01:43:28.312 DEBUG:teuthology.orchestra.run.smithi078:> 2024-04-25T01:43:28.312 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2024-04-25T01:43:28.312 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T01:43:28.312 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2024-04-25T01:43:28.313 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T01:43:28.313 DEBUG:teuthology.orchestra.run.smithi078:> 2024-04-25T01:43:28.313 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2024-04-25T01:43:28.313 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2024-04-25T01:43:28.313 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2024-04-25T01:43:28.313 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2024-04-25T01:43:28.313 DEBUG:teuthology.orchestra.run.smithi078:> 2024-04-25T01:43:28.313 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2024-04-25T01:43:28.313 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T01:43:28.313 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T01:43:28.313 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T01:43:28.313 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2024-04-25T01:43:28.313 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2024-04-25T01:43:28.314 DEBUG:teuthology.orchestra.run.smithi078:> return f 2024-04-25T01:43:28.314 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T01:43:28.314 DEBUG:teuthology.orchestra.run.smithi078:> 2024-04-25T01:43:28.314 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.admin")) 2024-04-25T01:43:28.314 DEBUG:teuthology.orchestra.run.smithi078:> ' 2024-04-25T01:43:28.429 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.admin.70628.asok 2024-04-25T01:43:28.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:43:28.440 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.70628.asok 2024-04-25T01:43:28.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:43:28.440 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.70628.asok status 2024-04-25T01:43:28.573 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:43:28.573 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2024-04-25T01:43:28.573 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T01:43:28.573 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:43:28.573 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "admin", 2024-04-25T01:43:28.573 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2024-04-25T01:43:28.573 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T01:43:28.573 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "70628", 2024-04-25T01:43:28.573 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2024-04-25T01:43:28.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:43:28.574 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2024-04-25T01:43:28.574 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2024-04-25T01:43:28.574 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 14790, 2024-04-25T01:43:28.574 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2024-04-25T01:43:28.574 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2024-04-25T01:43:28.574 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2024-04-25T01:43:28.574 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 14790 2024-04-25T01:43:28.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:43:28.574 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2024-04-25T01:43:28.574 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-04-25T01:43:28.574 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:43:28.574 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 724333285 2024-04-25T01:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2024-04-25T01:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-04-25T01:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 724333285 2024-04-25T01:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.14790 192.168.0.1:0/724333285", 2024-04-25T01:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/724333285", 2024-04-25T01:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2024-04-25T01:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 17, 2024-04-25T01:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 90, 2024-04-25T01:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2024-04-25T01:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2024-04-25T01:43:28.576 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2024-04-25T01:43:28.576 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:43:28.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:43:28.581 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2024-04-25T01:43:28.581 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:43:28.596 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2024-04-25T01:43:28.596 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:43:28.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:43:28.597 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:43:28.661 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T01:43:28.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T01:43:28.768 INFO:teuthology.orchestra.run.smithi078.stdout:# file: . 2024-04-25T01:43:28.768 INFO:teuthology.orchestra.run.smithi078.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T01:43:28.768 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:28.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:43:28.775 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:43:28.775 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:43:28.790 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:43:28.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.938+0000 7efc4855c640 1 -- 172.21.15.78:0/2338648209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc40101790 msgr2=0x7efc40101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:28.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.938+0000 7efc4855c640 1 --2- 172.21.15.78:0/2338648209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc40101790 0x7efc40101bb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009900 tx=0x7efc3c02a410 comp rx=0 tx=0).stop 2024-04-25T01:43:28.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.938+0000 7efc4855c640 1 -- 172.21.15.78:0/2338648209 shutdown_connections 2024-04-25T01:43:28.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.938+0000 7efc4855c640 1 --2- 172.21.15.78:0/2338648209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc40103340 0x7efc40109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:28.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.938+0000 7efc4855c640 1 --2- 172.21.15.78:0/2338648209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc40102980 0x7efc40102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:28.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.938+0000 7efc4855c640 1 --2- 172.21.15.78:0/2338648209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc40101790 0x7efc40101bb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:28.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.938+0000 7efc4855c640 1 -- 172.21.15.78:0/2338648209 >> 172.21.15.78:0/2338648209 conn(0x7efc400fcd20 msgr2=0x7efc400ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:28.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.938+0000 7efc4855c640 1 -- 172.21.15.78:0/2338648209 shutdown_connections 2024-04-25T01:43:28.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.938+0000 7efc4855c640 1 -- 172.21.15.78:0/2338648209 wait complete. 2024-04-25T01:43:28.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc4855c640 1 Processor -- start 2024-04-25T01:43:28.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc4855c640 1 -- start start 2024-04-25T01:43:28.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc4855c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc40101790 0x7efc40197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:28.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc4855c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc40102980 0x7efc40197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:28.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc4855c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc40103340 0x7efc4019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:28.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc4855c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc4010d430 con 0x7efc40102980 2024-04-25T01:43:28.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc4855c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efc4010d2b0 con 0x7efc40103340 2024-04-25T01:43:28.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc4855c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efc4010d5b0 con 0x7efc40101790 2024-04-25T01:43:28.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc45af7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc40102980 0x7efc40197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:28.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc45af7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc40102980 0x7efc40197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38750/0 (socket says 172.21.15.78:38750) 2024-04-25T01:43:28.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc45af7640 1 -- 172.21.15.78:0/2042411060 learned_addr learned my addr 172.21.15.78:0/2042411060 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:28.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc462f8640 1 --2- 172.21.15.78:0/2042411060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc40101790 0x7efc40197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:28.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc45af7640 1 -- 172.21.15.78:0/2042411060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc40101790 msgr2=0x7efc40197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:28.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc45af7640 1 --2- 172.21.15.78:0/2042411060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc40101790 0x7efc40197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:28.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc45af7640 1 -- 172.21.15.78:0/2042411060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc40103340 msgr2=0x7efc4019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:28.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc45af7640 1 --2- 172.21.15.78:0/2042411060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc40103340 0x7efc4019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:28.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc45af7640 1 -- 172.21.15.78:0/2042411060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4019f6d0 con 0x7efc40102980 2024-04-25T01:43:28.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc462f8640 1 --2- 172.21.15.78:0/2042411060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc40101790 0x7efc40197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:28.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.939+0000 7efc45af7640 1 --2- 172.21.15.78:0/2042411060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc40102980 0x7efc40197ff0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc3000d950 tx=0x7efc3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:28.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.940+0000 7efc377fe640 1 -- 172.21.15.78:0/2042411060 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30017070 con 0x7efc40102980 2024-04-25T01:43:28.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.940+0000 7efc4855c640 1 -- 172.21.15.78:0/2042411060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4019f9c0 con 0x7efc40102980 2024-04-25T01:43:28.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.940+0000 7efc377fe640 1 -- 172.21.15.78:0/2042411060 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc30009c30 con 0x7efc40102980 2024-04-25T01:43:28.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.940+0000 7efc377fe640 1 -- 172.21.15.78:0/2042411060 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3000f6b0 con 0x7efc40102980 2024-04-25T01:43:28.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.940+0000 7efc4855c640 1 -- 172.21.15.78:0/2042411060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4019ff60 con 0x7efc40102980 2024-04-25T01:43:28.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.941+0000 7efc4855c640 1 -- 172.21.15.78:0/2042411060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc40108b30 con 0x7efc40102980 2024-04-25T01:43:28.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.942+0000 7efc377fe640 1 -- 172.21.15.78:0/2042411060 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efc300109d0 con 0x7efc40102980 2024-04-25T01:43:28.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.942+0000 7efc377fe640 1 --2- 172.21.15.78:0/2042411060 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efc2c0a2b90 0x7efc2c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:28.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.943+0000 7efc462f8640 1 --2- 172.21.15.78:0/2042411060 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efc2c0a2b90 0x7efc2c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:28.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.943+0000 7efc377fe640 1 -- 172.21.15.78:0/2042411060 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7efc300cefa0 con 0x7efc40102980 2024-04-25T01:43:28.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.943+0000 7efc462f8640 1 --2- 172.21.15.78:0/2042411060 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efc2c0a2b90 0x7efc2c0a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc3c02aa00 tx=0x7efc3c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:28.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:28.947+0000 7efc377fe640 1 -- 172.21.15.78:0/2042411060 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efc30009de0 con 0x7efc40102980 2024-04-25T01:43:29.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.127+0000 7efc4855c640 1 -- 172.21.15.78:0/2042411060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc40062340 con 0x7efc40102980 2024-04-25T01:43:29.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.128+0000 7efc377fe640 1 -- 172.21.15.78:0/2042411060 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24151 (secure 0 0 0) 0x7efc3009bcc0 con 0x7efc40102980 2024-04-25T01:43:29.119 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:29.119 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:43:21.781660+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:43:29.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.130+0000 7efc4855c640 1 -- 172.21.15.78:0/2042411060 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efc2c0a2b90 msgr2=0x7efc2c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.130+0000 7efc4855c640 1 --2- 172.21.15.78:0/2042411060 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efc2c0a2b90 0x7efc2c0a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc3c02aa00 tx=0x7efc3c000bc0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.130+0000 7efc4855c640 1 -- 172.21.15.78:0/2042411060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc40102980 msgr2=0x7efc40197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.130+0000 7efc4855c640 1 --2- 172.21.15.78:0/2042411060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc40102980 0x7efc40197ff0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc3000d950 tx=0x7efc3000de10 comp rx=0 tx=0).stop 2024-04-25T01:43:29.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.130+0000 7efc4855c640 1 -- 172.21.15.78:0/2042411060 shutdown_connections 2024-04-25T01:43:29.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.130+0000 7efc4855c640 1 --2- 172.21.15.78:0/2042411060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc40103340 0x7efc4019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.130+0000 7efc4855c640 1 --2- 172.21.15.78:0/2042411060 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efc2c0a2b90 0x7efc2c0a5050 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.130+0000 7efc4855c640 1 --2- 172.21.15.78:0/2042411060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc40102980 0x7efc40197ff0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.130+0000 7efc4855c640 1 --2- 172.21.15.78:0/2042411060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc40101790 0x7efc40197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.130+0000 7efc4855c640 1 -- 172.21.15.78:0/2042411060 >> 172.21.15.78:0/2042411060 conn(0x7efc400fcd20 msgr2=0x7efc40105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:29.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.130+0000 7efc4855c640 1 -- 172.21.15.78:0/2042411060 shutdown_connections 2024-04-25T01:43:29.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.130+0000 7efc4855c640 1 -- 172.21.15.78:0/2042411060 wait complete. 2024-04-25T01:43:29.132 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:38:39.136418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:43:15.613881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:43:16.600270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:43:19.650457+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:43:29.132 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:43:29.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.242+0000 7f0ab5612640 1 -- 172.21.15.78:0/1224623030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ab01006a0 msgr2=0x7f0ab0100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.242+0000 7f0ab5612640 1 --2- 172.21.15.78:0/1224623030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ab01006a0 0x7f0ab0100ac0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009ca0 tx=0x7f0aa002a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.242+0000 7f0ab5612640 1 -- 172.21.15.78:0/1224623030 shutdown_connections 2024-04-25T01:43:29.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.242+0000 7f0ab5612640 1 --2- 172.21.15.78:0/1224623030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ab0102250 0x7f0ab0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.242+0000 7f0ab5612640 1 --2- 172.21.15.78:0/1224623030 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ab0101890 0x7f0ab0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.242+0000 7f0ab5612640 1 --2- 172.21.15.78:0/1224623030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ab01006a0 0x7f0ab0100ac0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.242+0000 7f0ab5612640 1 -- 172.21.15.78:0/1224623030 >> 172.21.15.78:0/1224623030 conn(0x7f0ab00fbc30 msgr2=0x7f0ab00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:29.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.242+0000 7f0ab5612640 1 -- 172.21.15.78:0/1224623030 shutdown_connections 2024-04-25T01:43:29.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.242+0000 7f0ab5612640 1 -- 172.21.15.78:0/1224623030 wait complete. 2024-04-25T01:43:29.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0ab5612640 1 Processor -- start 2024-04-25T01:43:29.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0ab5612640 1 -- start start 2024-04-25T01:43:29.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0ab5612640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ab01006a0 0x7f0ab0197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:29.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0ab5612640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ab0101890 0x7f0ab0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:29.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0ab5612640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ab0102250 0x7f0ab019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:29.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0ab5612640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab010c3c0 con 0x7f0ab0101890 2024-04-25T01:43:29.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0ab5612640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab010c240 con 0x7f0ab0102250 2024-04-25T01:43:29.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0ab5612640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab010c540 con 0x7f0ab01006a0 2024-04-25T01:43:29.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0aae7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ab0101890 0x7f0ab0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:29.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0aae7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ab0101890 0x7f0ab0198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38770/0 (socket says 172.21.15.78:38770) 2024-04-25T01:43:29.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0aae7fc640 1 -- 172.21.15.78:0/3949113555 learned_addr learned my addr 172.21.15.78:0/3949113555 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:29.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0aaeffd640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ab01006a0 0x7f0ab0197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:29.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0aaf7fe640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ab0102250 0x7f0ab019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:29.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0aae7fc640 1 -- 172.21.15.78:0/3949113555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ab01006a0 msgr2=0x7f0ab0197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0aae7fc640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ab01006a0 0x7f0ab0197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0aae7fc640 1 -- 172.21.15.78:0/3949113555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ab0102250 msgr2=0x7f0ab019f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0aae7fc640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ab0102250 0x7f0ab019f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0aae7fc640 1 -- 172.21.15.78:0/3949113555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab019f7c0 con 0x7f0ab0101890 2024-04-25T01:43:29.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0aaeffd640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ab01006a0 0x7f0ab0197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:29.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.243+0000 7f0aae7fc640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ab0101890 0x7f0ab0198130 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400b4a0 tx=0x7f0aa400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:29.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.244+0000 7f0a97fff640 1 -- 172.21.15.78:0/3949113555 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4014070 con 0x7f0ab0101890 2024-04-25T01:43:29.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.244+0000 7f0ab5612640 1 -- 172.21.15.78:0/3949113555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab019fab0 con 0x7f0ab0101890 2024-04-25T01:43:29.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.244+0000 7f0a97fff640 1 -- 172.21.15.78:0/3949113555 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0aa4010af0 con 0x7f0ab0101890 2024-04-25T01:43:29.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.244+0000 7f0a97fff640 1 -- 172.21.15.78:0/3949113555 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4017d10 con 0x7f0ab0101890 2024-04-25T01:43:29.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.244+0000 7f0ab5612640 1 -- 172.21.15.78:0/3949113555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab01a0000 con 0x7f0ab0101890 2024-04-25T01:43:29.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.245+0000 7f0ab5612640 1 -- 172.21.15.78:0/3949113555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab01a0290 con 0x7f0ab0101890 2024-04-25T01:43:29.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.246+0000 7f0a97fff640 1 -- 172.21.15.78:0/3949113555 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0aa4017070 con 0x7f0ab0101890 2024-04-25T01:43:29.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.246+0000 7f0a97fff640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0a980a2b90 0x7f0a980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:29.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.246+0000 7f0a97fff640 1 -- 172.21.15.78:0/3949113555 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0aa40ce6a0 con 0x7f0ab0101890 2024-04-25T01:43:29.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.246+0000 7f0aaeffd640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0a980a2b90 0x7f0a980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:29.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.247+0000 7f0aaeffd640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0a980a2b90 0x7f0a980a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0006010 tx=0x7f0aa002f3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:29.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.251+0000 7f0a97fff640 1 -- 172.21.15.78:0/3949113555 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0aa409b3c0 con 0x7f0ab0101890 2024-04-25T01:43:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.429+0000 7f0ab5612640 1 -- 172.21.15.78:0/3949113555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ab01a0460 con 0x7f0ab0101890 2024-04-25T01:43:29.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.430+0000 7f0a97fff640 1 -- 172.21.15.78:0/3949113555 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f0aa409acc0 con 0x7f0ab0101890 2024-04-25T01:43:29.420 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T01:43:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.432+0000 7f0ab5612640 1 -- 172.21.15.78:0/3949113555 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0a980a2b90 msgr2=0x7f0a980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.432+0000 7f0ab5612640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0a980a2b90 0x7f0a980a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0006010 tx=0x7f0aa002f3c0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.432+0000 7f0ab5612640 1 -- 172.21.15.78:0/3949113555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ab0101890 msgr2=0x7f0ab0198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.432+0000 7f0ab5612640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ab0101890 0x7f0ab0198130 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400b4a0 tx=0x7f0aa400b960 comp rx=0 tx=0).stop 2024-04-25T01:43:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.432+0000 7f0ab5612640 1 -- 172.21.15.78:0/3949113555 shutdown_connections 2024-04-25T01:43:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.432+0000 7f0ab5612640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ab0102250 0x7f0ab019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.432+0000 7f0ab5612640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0a980a2b90 0x7f0a980a5050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.432+0000 7f0ab5612640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ab0101890 0x7f0ab0198130 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.432+0000 7f0ab5612640 1 --2- 172.21.15.78:0/3949113555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ab01006a0 0x7f0ab0197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.432+0000 7f0ab5612640 1 -- 172.21.15.78:0/3949113555 >> 172.21.15.78:0/3949113555 conn(0x7f0ab00fbc30 msgr2=0x7f0ab0069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:29.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.432+0000 7f0ab5612640 1 -- 172.21.15.78:0/3949113555 shutdown_connections 2024-04-25T01:43:29.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.432+0000 7f0ab5612640 1 -- 172.21.15.78:0/3949113555 wait complete. 2024-04-25T01:43:29.434 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:43:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.545+0000 7f0d9e509640 1 -- 172.21.15.78:0/1197433246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d98102220 msgr2=0x7f0d98104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.545+0000 7f0d9e509640 1 --2- 172.21.15.78:0/1197433246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d98102220 0x7f0d98104610 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009900 tx=0x7f0d8002a410 comp rx=0 tx=0).stop 2024-04-25T01:43:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.545+0000 7f0d9e509640 1 -- 172.21.15.78:0/1197433246 shutdown_connections 2024-04-25T01:43:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.545+0000 7f0d9e509640 1 --2- 172.21.15.78:0/1197433246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d98107480 0x7f0d981098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.545+0000 7f0d9e509640 1 --2- 172.21.15.78:0/1197433246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d98104b50 0x7f0d98106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.545+0000 7f0d9e509640 1 --2- 172.21.15.78:0/1197433246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d98102220 0x7f0d98104610 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.545+0000 7f0d9e509640 1 -- 172.21.15.78:0/1197433246 >> 172.21.15.78:0/1197433246 conn(0x7f0d980fbc30 msgr2=0x7f0d980fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.545+0000 7f0d9e509640 1 -- 172.21.15.78:0/1197433246 shutdown_connections 2024-04-25T01:43:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.545+0000 7f0d9e509640 1 -- 172.21.15.78:0/1197433246 wait complete. 2024-04-25T01:43:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.545+0000 7f0d9e509640 1 Processor -- start 2024-04-25T01:43:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.545+0000 7f0d9e509640 1 -- start start 2024-04-25T01:43:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d9e509640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d98102220 0x7f0d98101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d9e509640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d98104b50 0x7f0d981001d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d9e509640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d98107480 0x7f0d98100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d9e509640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9810d490 con 0x7f0d98104b50 2024-04-25T01:43:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d9e509640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9810d310 con 0x7f0d98102220 2024-04-25T01:43:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d9e509640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9810d610 con 0x7f0d98107480 2024-04-25T01:43:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d98104b50 0x7f0d981001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d98104b50 0x7f0d981001d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49218/0 (socket says 172.21.15.78:49218) 2024-04-25T01:43:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d977fe640 1 -- 172.21.15.78:0/1280902745 learned_addr learned my addr 172.21.15.78:0/1280902745 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d9caa6640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d98107480 0x7f0d98100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d97fff640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d98102220 0x7f0d98101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d977fe640 1 -- 172.21.15.78:0/1280902745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d98107480 msgr2=0x7f0d98100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d977fe640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d98107480 0x7f0d98100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d977fe640 1 -- 172.21.15.78:0/1280902745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d98102220 msgr2=0x7f0d98101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d977fe640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d98102220 0x7f0d98101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d977fe640 1 -- 172.21.15.78:0/1280902745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d98100fa0 con 0x7f0d98104b50 2024-04-25T01:43:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d97fff640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d98102220 0x7f0d98101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d977fe640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d98104b50 0x7f0d981001d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800e9c0 tx=0x7f0d8800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d957fa640 1 -- 172.21.15.78:0/1280902745 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8800cd80 con 0x7f0d98104b50 2024-04-25T01:43:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d9e509640 1 -- 172.21.15.78:0/1280902745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d981a1c20 con 0x7f0d98104b50 2024-04-25T01:43:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d957fa640 1 -- 172.21.15.78:0/1280902745 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d88005500 con 0x7f0d98104b50 2024-04-25T01:43:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.546+0000 7f0d957fa640 1 -- 172.21.15.78:0/1280902745 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88010690 con 0x7f0d98104b50 2024-04-25T01:43:29.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.547+0000 7f0d9e509640 1 -- 172.21.15.78:0/1280902745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d981a21c0 con 0x7f0d98104b50 2024-04-25T01:43:29.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.548+0000 7f0d9e509640 1 -- 172.21.15.78:0/1280902745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d980fd720 con 0x7f0d98104b50 2024-04-25T01:43:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.549+0000 7f0d957fa640 1 -- 172.21.15.78:0/1280902745 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0d880046c0 con 0x7f0d98104b50 2024-04-25T01:43:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.549+0000 7f0d957fa640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0d840a2b90 0x7f0d840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.549+0000 7f0d97fff640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0d840a2b90 0x7f0d840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.550+0000 7f0d957fa640 1 -- 172.21.15.78:0/1280902745 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0d880cf710 con 0x7f0d98104b50 2024-04-25T01:43:29.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.550+0000 7f0d97fff640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0d840a2b90 0x7f0d840a5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0d80006010 tx=0x7f0d80000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:29.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.554+0000 7f0d957fa640 1 -- 172.21.15.78:0/1280902745 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0d8809c430 con 0x7f0d98104b50 2024-04-25T01:43:29.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.729+0000 7f0d9e509640 1 -- 172.21.15.78:0/1280902745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0d98062340 con 0x7f0d98104b50 2024-04-25T01:43:29.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.729+0000 7f0d957fa640 1 -- 172.21.15.78:0/1280902745 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f0d8809bd30 con 0x7f0d98104b50 2024-04-25T01:43:29.720 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:43:29.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.731+0000 7f0d9e509640 1 -- 172.21.15.78:0/1280902745 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0d840a2b90 msgr2=0x7f0d840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.731+0000 7f0d9e509640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0d840a2b90 0x7f0d840a5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0d80006010 tx=0x7f0d80000bc0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.731+0000 7f0d9e509640 1 -- 172.21.15.78:0/1280902745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d98104b50 msgr2=0x7f0d981001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.731+0000 7f0d9e509640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d98104b50 0x7f0d981001d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800e9c0 tx=0x7f0d8800ee80 comp rx=0 tx=0).stop 2024-04-25T01:43:29.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.731+0000 7f0d9e509640 1 -- 172.21.15.78:0/1280902745 shutdown_connections 2024-04-25T01:43:29.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.731+0000 7f0d9e509640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d98107480 0x7f0d98100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.731+0000 7f0d9e509640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0d840a2b90 0x7f0d840a5050 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.731+0000 7f0d9e509640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d98104b50 0x7f0d981001d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.731+0000 7f0d9e509640 1 --2- 172.21.15.78:0/1280902745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d98102220 0x7f0d98101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.731+0000 7f0d9e509640 1 -- 172.21.15.78:0/1280902745 >> 172.21.15.78:0/1280902745 conn(0x7f0d980fbc30 msgr2=0x7f0d98102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:29.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.731+0000 7f0d9e509640 1 -- 172.21.15.78:0/1280902745 shutdown_connections 2024-04-25T01:43:29.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.731+0000 7f0d9e509640 1 -- 172.21.15.78:0/1280902745 wait complete. 2024-04-25T01:43:29.733 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:43:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.843+0000 7fca08b22640 1 -- 172.21.15.78:0/1011825170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca041033e0 msgr2=0x7fca04109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.843+0000 7fca08b22640 1 --2- 172.21.15.78:0/1011825170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca041033e0 0x7fca04109c70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc009cd0 tx=0x7fc9fc02a8e0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.843+0000 7fca08b22640 1 -- 172.21.15.78:0/1011825170 shutdown_connections 2024-04-25T01:43:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.843+0000 7fca08b22640 1 --2- 172.21.15.78:0/1011825170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca041033e0 0x7fca04109c70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.843+0000 7fca08b22640 1 --2- 172.21.15.78:0/1011825170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca04102a20 0x7fca04102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.843+0000 7fca08b22640 1 --2- 172.21.15.78:0/1011825170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca040ffa50 0x7fca040ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.843+0000 7fca08b22640 1 -- 172.21.15.78:0/1011825170 >> 172.21.15.78:0/1011825170 conn(0x7fca040fb380 msgr2=0x7fca040fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:29.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.843+0000 7fca08b22640 1 -- 172.21.15.78:0/1011825170 shutdown_connections 2024-04-25T01:43:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.843+0000 7fca08b22640 1 -- 172.21.15.78:0/1011825170 wait complete. 2024-04-25T01:43:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.843+0000 7fca08b22640 1 Processor -- start 2024-04-25T01:43:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.843+0000 7fca08b22640 1 -- start start 2024-04-25T01:43:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca08b22640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca040ffa50 0x7fca04197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca08b22640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca04102a20 0x7fca04197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca08b22640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca041033e0 0x7fca0419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca08b22640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fca0410d430 con 0x7fca041033e0 2024-04-25T01:43:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca08b22640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fca0410d2b0 con 0x7fca040ffa50 2024-04-25T01:43:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca08b22640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca0410d5b0 con 0x7fca04102a20 2024-04-25T01:43:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca02d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca041033e0 0x7fca0419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca02d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca041033e0 0x7fca0419efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49236/0 (socket says 172.21.15.78:49236) 2024-04-25T01:43:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca02d9d640 1 -- 172.21.15.78:0/153939415 learned_addr learned my addr 172.21.15.78:0/153939415 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca0259c640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca040ffa50 0x7fca04197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:29.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca01d9b640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca04102a20 0x7fca04197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca02d9d640 1 -- 172.21.15.78:0/153939415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca04102a20 msgr2=0x7fca04197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca02d9d640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca04102a20 0x7fca04197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca02d9d640 1 -- 172.21.15.78:0/153939415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca040ffa50 msgr2=0x7fca04197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca02d9d640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca040ffa50 0x7fca04197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca02d9d640 1 -- 172.21.15.78:0/153939415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0419f630 con 0x7fca041033e0 2024-04-25T01:43:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca0259c640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca040ffa50 0x7fca04197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca01d9b640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca04102a20 0x7fca04197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:43:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca02d9d640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca041033e0 0x7fca0419efc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc004af0 tx=0x7fc9fc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fc9f37fe640 1 -- 172.21.15.78:0/153939415 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc004450 con 0x7fca041033e0 2024-04-25T01:43:29.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fc9f37fe640 1 -- 172.21.15.78:0/153939415 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9fc0045f0 con 0x7fca041033e0 2024-04-25T01:43:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca08b22640 1 -- 172.21.15.78:0/153939415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0419f8c0 con 0x7fca041033e0 2024-04-25T01:43:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fc9f37fe640 1 -- 172.21.15.78:0/153939415 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc0466c0 con 0x7fca041033e0 2024-04-25T01:43:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.844+0000 7fca08b22640 1 -- 172.21.15.78:0/153939415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca0419fe10 con 0x7fca041033e0 2024-04-25T01:43:29.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.845+0000 7fca08b22640 1 -- 172.21.15.78:0/153939415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca04108a80 con 0x7fca041033e0 2024-04-25T01:43:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.847+0000 7fc9f37fe640 1 -- 172.21.15.78:0/153939415 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc9fc046860 con 0x7fca041033e0 2024-04-25T01:43:29.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.847+0000 7fc9f37fe640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc9ec0a2b90 0x7fc9ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.847+0000 7fca0259c640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc9ec0a2b90 0x7fc9ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.847+0000 7fc9f37fe640 1 -- 172.21.15.78:0/153939415 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc9fc02e030 con 0x7fca041033e0 2024-04-25T01:43:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.848+0000 7fca0259c640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc9ec0a2b90 0x7fc9ec0a5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fca04069bc0 tx=0x7fc9f400a5e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:29.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:29.851+0000 7fc9f37fe640 1 -- 172.21.15.78:0/153939415 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc9fc04b060 con 0x7fca041033e0 2024-04-25T01:43:30.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:29 smithi078 ceph-mon[29469]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-25T01:43:30.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:29 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2042411060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:30.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:29 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3949113555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:43:30.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:29 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1280902745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:43:30.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.028+0000 7fca08b22640 1 -- 172.21.15.78:0/153939415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fca0402d3c0 con 0x7fca041033e0 2024-04-25T01:43:30.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.028+0000 7fc9f37fe640 1 -- 172.21.15.78:0/153939415 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fc9fc0bdae0 con 0x7fca041033e0 2024-04-25T01:43:30.019 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T01:43:30.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.030+0000 7fca08b22640 1 -- 172.21.15.78:0/153939415 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc9ec0a2b90 msgr2=0x7fc9ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.030+0000 7fca08b22640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc9ec0a2b90 0x7fc9ec0a5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fca04069bc0 tx=0x7fc9f400a5e0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.030+0000 7fca08b22640 1 -- 172.21.15.78:0/153939415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca041033e0 msgr2=0x7fca0419efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.030+0000 7fca08b22640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca041033e0 0x7fca0419efc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc004af0 tx=0x7fc9fc003c10 comp rx=0 tx=0).stop 2024-04-25T01:43:30.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.030+0000 7fca08b22640 1 -- 172.21.15.78:0/153939415 shutdown_connections 2024-04-25T01:43:30.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.030+0000 7fca08b22640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc9ec0a2b90 0x7fc9ec0a5050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.030+0000 7fca08b22640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca041033e0 0x7fca0419efc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.030+0000 7fca08b22640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca04102a20 0x7fca04197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.030+0000 7fca08b22640 1 --2- 172.21.15.78:0/153939415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca040ffa50 0x7fca04197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.030+0000 7fca08b22640 1 -- 172.21.15.78:0/153939415 >> 172.21.15.78:0/153939415 conn(0x7fca040fb380 msgr2=0x7fca04105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:30.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.030+0000 7fca08b22640 1 -- 172.21.15.78:0/153939415 shutdown_connections 2024-04-25T01:43:30.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.030+0000 7fca08b22640 1 -- 172.21.15.78:0/153939415 wait complete. 2024-04-25T01:43:30.032 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:43:30.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 -- 172.21.15.78:0/641771833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f20102230 msgr2=0x7f2f20108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 --2- 172.21.15.78:0/641771833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f20102230 0x7f2f20108ac0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009ca0 tx=0x7f2f1802a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 -- 172.21.15.78:0/641771833 shutdown_connections 2024-04-25T01:43:30.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 --2- 172.21.15.78:0/641771833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f20102230 0x7f2f20108ac0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 --2- 172.21.15.78:0/641771833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f20101870 0x7f2f20101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 --2- 172.21.15.78:0/641771833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f20100680 0x7f2f20100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 -- 172.21.15.78:0/641771833 >> 172.21.15.78:0/641771833 conn(0x7f2f200fbbf0 msgr2=0x7f2f200fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:30.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 -- 172.21.15.78:0/641771833 shutdown_connections 2024-04-25T01:43:30.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 -- 172.21.15.78:0/641771833 wait complete. 2024-04-25T01:43:30.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 Processor -- start 2024-04-25T01:43:30.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 -- start start 2024-04-25T01:43:30.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f20100680 0x7f2f20196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:30.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f20101870 0x7f2f20196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:30.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f20102230 0x7f2f2019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:30.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2010c300 con 0x7f2f20101870 2024-04-25T01:43:30.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2010c180 con 0x7f2f20102230 2024-04-25T01:43:30.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.143+0000 7f2f265f9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2010c480 con 0x7f2f20100680 2024-04-25T01:43:30.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f20101870 0x7f2f20196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:30.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f20101870 0x7f2f20196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49244/0 (socket says 172.21.15.78:49244) 2024-04-25T01:43:30.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f1f7fe640 1 -- 172.21.15.78:0/968127596 learned_addr learned my addr 172.21.15.78:0/968127596 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:30.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f24b96640 1 --2- 172.21.15.78:0/968127596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f20102230 0x7f2f2019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:30.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f1ffff640 1 --2- 172.21.15.78:0/968127596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f20100680 0x7f2f20196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:30.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f1f7fe640 1 -- 172.21.15.78:0/968127596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f20100680 msgr2=0x7f2f20196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f1f7fe640 1 --2- 172.21.15.78:0/968127596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f20100680 0x7f2f20196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f1f7fe640 1 -- 172.21.15.78:0/968127596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f20102230 msgr2=0x7f2f2019df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f1f7fe640 1 --2- 172.21.15.78:0/968127596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f20102230 0x7f2f2019df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f1f7fe640 1 -- 172.21.15.78:0/968127596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2019e590 con 0x7f2f20101870 2024-04-25T01:43:30.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f1f7fe640 1 --2- 172.21.15.78:0/968127596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f20101870 0x7f2f20196eb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400eb00 tx=0x7f2f14009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:30.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f1d7fa640 1 -- 172.21.15.78:0/968127596 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1401c070 con 0x7f2f20101870 2024-04-25T01:43:30.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f265f9640 1 -- 172.21.15.78:0/968127596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f20068930 con 0x7f2f20101870 2024-04-25T01:43:30.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f265f9640 1 -- 172.21.15.78:0/968127596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f20068ed0 con 0x7f2f20101870 2024-04-25T01:43:30.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f1d7fa640 1 -- 172.21.15.78:0/968127596 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1400bc30 con 0x7f2f20101870 2024-04-25T01:43:30.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.144+0000 7f2f1d7fa640 1 -- 172.21.15.78:0/968127596 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14020610 con 0x7f2f20101870 2024-04-25T01:43:30.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.145+0000 7f2f265f9640 1 -- 172.21.15.78:0/968127596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f0c0052c0 con 0x7f2f20101870 2024-04-25T01:43:30.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.146+0000 7f2f1d7fa640 1 -- 172.21.15.78:0/968127596 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2f1401f070 con 0x7f2f20101870 2024-04-25T01:43:30.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.147+0000 7f2f1d7fa640 1 --2- 172.21.15.78:0/968127596 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2f080a2b90 0x7f2f080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:30.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.147+0000 7f2f1d7fa640 1 -- 172.21.15.78:0/968127596 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2f140d0840 con 0x7f2f20101870 2024-04-25T01:43:30.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.147+0000 7f2f1ffff640 1 --2- 172.21.15.78:0/968127596 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2f080a2b90 0x7f2f080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:30.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.147+0000 7f2f1ffff640 1 --2- 172.21.15.78:0/968127596 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2f080a2b90 0x7f2f080a5050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2f10005fd0 tx=0x7f2f10005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:30.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.151+0000 7f2f1d7fa640 1 -- 172.21.15.78:0/968127596 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2f1409d560 con 0x7f2f20101870 2024-04-25T01:43:30.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:29 smithi152 ceph-mon[32465]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-25T01:43:30.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:29 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2042411060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:30.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:29 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3949113555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:43:30.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:29 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1280902745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:43:30.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.325+0000 7f2f265f9640 1 -- 172.21.15.78:0/968127596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2f0c005dc0 con 0x7f2f20101870 2024-04-25T01:43:30.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.326+0000 7f2f1d7fa640 1 -- 172.21.15.78:0/968127596 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f2f1409ce60 con 0x7f2f20101870 2024-04-25T01:43:30.317 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:43:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.328+0000 7f2f265f9640 1 -- 172.21.15.78:0/968127596 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2f080a2b90 msgr2=0x7f2f080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.328+0000 7f2f265f9640 1 --2- 172.21.15.78:0/968127596 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2f080a2b90 0x7f2f080a5050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2f10005fd0 tx=0x7f2f10005850 comp rx=0 tx=0).stop 2024-04-25T01:43:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.328+0000 7f2f265f9640 1 -- 172.21.15.78:0/968127596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f20101870 msgr2=0x7f2f20196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.328+0000 7f2f265f9640 1 --2- 172.21.15.78:0/968127596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f20101870 0x7f2f20196eb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400eb00 tx=0x7f2f14009f20 comp rx=0 tx=0).stop 2024-04-25T01:43:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.328+0000 7f2f265f9640 1 -- 172.21.15.78:0/968127596 shutdown_connections 2024-04-25T01:43:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.328+0000 7f2f265f9640 1 --2- 172.21.15.78:0/968127596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f20102230 0x7f2f2019df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.328+0000 7f2f265f9640 1 --2- 172.21.15.78:0/968127596 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2f080a2b90 0x7f2f080a5050 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.328+0000 7f2f265f9640 1 --2- 172.21.15.78:0/968127596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f20101870 0x7f2f20196eb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.328+0000 7f2f265f9640 1 --2- 172.21.15.78:0/968127596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2f20100680 0x7f2f20196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.328+0000 7f2f265f9640 1 -- 172.21.15.78:0/968127596 >> 172.21.15.78:0/968127596 conn(0x7f2f200fbbf0 msgr2=0x7f2f20104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.328+0000 7f2f265f9640 1 -- 172.21.15.78:0/968127596 shutdown_connections 2024-04-25T01:43:30.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.328+0000 7f2f265f9640 1 -- 172.21.15.78:0/968127596 wait complete. 2024-04-25T01:43:30.329 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:43:30.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:29 smithi196 ceph-mon[32978]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-25T01:43:30.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:29 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2042411060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:30.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:29 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3949113555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:43:30.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:29 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1280902745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:43:30.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.437+0000 7f8426934640 1 -- 172.21.15.78:0/1024252115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84201006a0 msgr2=0x7f8420100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.437+0000 7f8426934640 1 --2- 172.21.15.78:0/1024252115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84201006a0 0x7f8420100ac0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8410009900 tx=0x7f841002a410 comp rx=0 tx=0).stop 2024-04-25T01:43:30.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.437+0000 7f8426934640 1 -- 172.21.15.78:0/1024252115 shutdown_connections 2024-04-25T01:43:30.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.437+0000 7f8426934640 1 --2- 172.21.15.78:0/1024252115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8420102250 0x7f8420108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.437+0000 7f8426934640 1 --2- 172.21.15.78:0/1024252115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8420101890 0x7f8420101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.437+0000 7f8426934640 1 --2- 172.21.15.78:0/1024252115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84201006a0 0x7f8420100ac0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.437+0000 7f8426934640 1 -- 172.21.15.78:0/1024252115 >> 172.21.15.78:0/1024252115 conn(0x7f84200fbc30 msgr2=0x7f84200fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.437+0000 7f8426934640 1 -- 172.21.15.78:0/1024252115 shutdown_connections 2024-04-25T01:43:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.437+0000 7f8426934640 1 -- 172.21.15.78:0/1024252115 wait complete. 2024-04-25T01:43:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.438+0000 7f8426934640 1 Processor -- start 2024-04-25T01:43:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.438+0000 7f8426934640 1 -- start start 2024-04-25T01:43:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.438+0000 7f8426934640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84201006a0 0x7f8420199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.438+0000 7f8426934640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420101890 0x7f842019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.438+0000 7f8426934640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8420102250 0x7f84201a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.438+0000 7f8426934640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f842010c320 con 0x7f8420101890 2024-04-25T01:43:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.438+0000 7f8426934640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f842010c1a0 con 0x7f8420102250 2024-04-25T01:43:30.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.438+0000 7f8426934640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f842010c4a0 con 0x7f84201006a0 2024-04-25T01:43:30.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.438+0000 7f841f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420101890 0x7f842019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:30.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.438+0000 7f841f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420101890 0x7f842019a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49260/0 (socket says 172.21.15.78:49260) 2024-04-25T01:43:30.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.438+0000 7f841f7fe640 1 -- 172.21.15.78:0/501757765 learned_addr learned my addr 172.21.15.78:0/501757765 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:30.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.438+0000 7f8424ed1640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8420102250 0x7f84201a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:30.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.439+0000 7f841ffff640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84201006a0 0x7f8420199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:30.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.439+0000 7f841f7fe640 1 -- 172.21.15.78:0/501757765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84201006a0 msgr2=0x7f8420199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.439+0000 7f841f7fe640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84201006a0 0x7f8420199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.439+0000 7f841f7fe640 1 -- 172.21.15.78:0/501757765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8420102250 msgr2=0x7f84201a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.439+0000 7f841f7fe640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8420102250 0x7f84201a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.439+0000 7f841f7fe640 1 -- 172.21.15.78:0/501757765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84201a1920 con 0x7f8420101890 2024-04-25T01:43:30.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.439+0000 7f841ffff640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84201006a0 0x7f8420199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:43:30.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.439+0000 7f841f7fe640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420101890 0x7f842019a2b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f841400e9f0 tx=0x7f841400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:30.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.439+0000 7f841d7fa640 1 -- 172.21.15.78:0/501757765 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841400cd80 con 0x7f8420101890 2024-04-25T01:43:30.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.439+0000 7f8426934640 1 -- 172.21.15.78:0/501757765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84201a1c10 con 0x7f8420101890 2024-04-25T01:43:30.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.439+0000 7f841d7fa640 1 -- 172.21.15.78:0/501757765 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8414010910 con 0x7f8420101890 2024-04-25T01:43:30.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.439+0000 7f841d7fa640 1 -- 172.21.15.78:0/501757765 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414018990 con 0x7f8420101890 2024-04-25T01:43:30.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.439+0000 7f8426934640 1 -- 172.21.15.78:0/501757765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84201a2160 con 0x7f8420101890 2024-04-25T01:43:30.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.440+0000 7f8426934640 1 -- 172.21.15.78:0/501757765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84201a2330 con 0x7f8420101890 2024-04-25T01:43:30.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.441+0000 7f841d7fa640 1 -- 172.21.15.78:0/501757765 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8414010ac0 con 0x7f8420101890 2024-04-25T01:43:30.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.441+0000 7f841d7fa640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f84080a2b90 0x7f84080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:30.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.442+0000 7f841ffff640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f84080a2b90 0x7f84080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:30.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.442+0000 7f841d7fa640 1 -- 172.21.15.78:0/501757765 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f84140cf620 con 0x7f8420101890 2024-04-25T01:43:30.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.442+0000 7f841ffff640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f84080a2b90 0x7f84080a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8410000c00 tx=0x7f8410031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:30.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.446+0000 7f841d7fa640 1 -- 172.21.15.78:0/501757765 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f841409c340 con 0x7f8420101890 2024-04-25T01:43:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.633+0000 7f8426934640 1 -- 172.21.15.78:0/501757765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f84201a26c0 con 0x7f8420101890 2024-04-25T01:43:30.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.633+0000 7f841d7fa640 1 -- 172.21.15.78:0/501757765 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+589 (secure 0 0 0) 0x7f841409bc40 con 0x7f8420101890 2024-04-25T01:43:30.624 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1101495738 2024-04-26T01:36:16.812502+0000 2024-04-25T01:43:30.624 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1101495738 2024-04-26T01:36:16.812502+0000 2024-04-25T01:43:30.625 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2762765405 2024-04-26T01:36:16.812502+0000 2024-04-25T01:43:30.625 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/4170716207 2024-04-26T01:35:50.507447+0000 2024-04-25T01:43:30.625 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1136116795 2024-04-26T01:35:50.507447+0000 2024-04-25T01:43:30.625 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1544860340 2024-04-26T01:35:50.507447+0000 2024-04-25T01:43:30.625 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1544860340 2024-04-26T01:35:50.507447+0000 2024-04-25T01:43:30.625 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/211357528 2024-04-26T01:36:16.812502+0000 2024-04-25T01:43:30.625 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/88994848 2024-04-26T01:36:16.812502+0000 2024-04-25T01:43:30.625 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1196975031 2024-04-26T01:35:50.507447+0000 2024-04-25T01:43:30.625 INFO:teuthology.orchestra.run.smithi078.stderr:listed 10 entries 2024-04-25T01:43:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.635+0000 7f8426934640 1 -- 172.21.15.78:0/501757765 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f84080a2b90 msgr2=0x7f84080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.635+0000 7f8426934640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f84080a2b90 0x7f84080a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8410000c00 tx=0x7f8410031400 comp rx=0 tx=0).stop 2024-04-25T01:43:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.635+0000 7f8426934640 1 -- 172.21.15.78:0/501757765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420101890 msgr2=0x7f842019a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.635+0000 7f8426934640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420101890 0x7f842019a2b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f841400e9f0 tx=0x7f841400eeb0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.635+0000 7f8426934640 1 -- 172.21.15.78:0/501757765 shutdown_connections 2024-04-25T01:43:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.635+0000 7f8426934640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8420102250 0x7f84201a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.635+0000 7f8426934640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f84080a2b90 0x7f84080a5050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.635+0000 7f8426934640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420101890 0x7f842019a2b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.635+0000 7f8426934640 1 --2- 172.21.15.78:0/501757765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84201006a0 0x7f8420199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.635+0000 7f8426934640 1 -- 172.21.15.78:0/501757765 >> 172.21.15.78:0/501757765 conn(0x7f84200fbc30 msgr2=0x7f84200fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.636+0000 7f8426934640 1 -- 172.21.15.78:0/501757765 shutdown_connections 2024-04-25T01:43:30.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.636+0000 7f8426934640 1 -- 172.21.15.78:0/501757765 wait complete. 2024-04-25T01:43:30.637 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:43:30.637 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:43:30.653 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:43:30.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.800+0000 7f8030231640 1 -- 172.21.15.78:0/3142775367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8028100680 msgr2=0x7f8028100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.800+0000 7f8030231640 1 --2- 172.21.15.78:0/3142775367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8028100680 0x7f8028100aa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8020009ca0 tx=0x7f802002a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.800+0000 7f8030231640 1 -- 172.21.15.78:0/3142775367 shutdown_connections 2024-04-25T01:43:30.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.800+0000 7f8030231640 1 --2- 172.21.15.78:0/3142775367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8028102230 0x7f8028108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.800+0000 7f8030231640 1 --2- 172.21.15.78:0/3142775367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8028101870 0x7f8028101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.800+0000 7f8030231640 1 --2- 172.21.15.78:0/3142775367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8028100680 0x7f8028100aa0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.800+0000 7f8030231640 1 -- 172.21.15.78:0/3142775367 >> 172.21.15.78:0/3142775367 conn(0x7f80280fbbf0 msgr2=0x7f80280fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:30.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.801+0000 7f8030231640 1 -- 172.21.15.78:0/3142775367 shutdown_connections 2024-04-25T01:43:30.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.801+0000 7f8030231640 1 -- 172.21.15.78:0/3142775367 wait complete. 2024-04-25T01:43:30.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.801+0000 7f8030231640 1 Processor -- start 2024-04-25T01:43:30.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.801+0000 7f8030231640 1 -- start start 2024-04-25T01:43:30.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.801+0000 7f8030231640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8028100680 0x7f8028069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:30.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.801+0000 7f8030231640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8028101870 0x7f802819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:30.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.801+0000 7f8030231640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8028102230 0x7f802819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:30.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.801+0000 7f8030231640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f802810c3d0 con 0x7f8028101870 2024-04-25T01:43:30.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.801+0000 7f8030231640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f802810c250 con 0x7f8028102230 2024-04-25T01:43:30.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.801+0000 7f8030231640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f802810c550 con 0x7f8028100680 2024-04-25T01:43:30.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.801+0000 7f802d7cc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8028101870 0x7f802819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:30.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.801+0000 7f802d7cc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8028101870 0x7f802819c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49274/0 (socket says 172.21.15.78:49274) 2024-04-25T01:43:30.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.801+0000 7f802d7cc640 1 -- 172.21.15.78:0/3225161457 learned_addr learned my addr 172.21.15.78:0/3225161457 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:30.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.802+0000 7f802e7ce640 1 --2- 172.21.15.78:0/3225161457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8028102230 0x7f802819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:30.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.802+0000 7f802dfcd640 1 --2- 172.21.15.78:0/3225161457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8028100680 0x7f8028069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:30.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.802+0000 7f802d7cc640 1 -- 172.21.15.78:0/3225161457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8028100680 msgr2=0x7f8028069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.802+0000 7f802d7cc640 1 --2- 172.21.15.78:0/3225161457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8028100680 0x7f8028069460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.802+0000 7f802d7cc640 1 -- 172.21.15.78:0/3225161457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8028102230 msgr2=0x7f802819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.802+0000 7f802d7cc640 1 --2- 172.21.15.78:0/3225161457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8028102230 0x7f802819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.802+0000 7f802d7cc640 1 -- 172.21.15.78:0/3225161457 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802819f830 con 0x7f8028101870 2024-04-25T01:43:30.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.802+0000 7f802d7cc640 1 --2- 172.21.15.78:0/3225161457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8028101870 0x7f802819c930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f801800b4a0 tx=0x7f801800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:30.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.802+0000 7f801effd640 1 -- 172.21.15.78:0/3225161457 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018014070 con 0x7f8028101870 2024-04-25T01:43:30.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.802+0000 7f801effd640 1 -- 172.21.15.78:0/3225161457 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8018010af0 con 0x7f8028101870 2024-04-25T01:43:30.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.802+0000 7f8030231640 1 -- 172.21.15.78:0/3225161457 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802819fb20 con 0x7f8028101870 2024-04-25T01:43:30.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.802+0000 7f801effd640 1 -- 172.21.15.78:0/3225161457 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018017d10 con 0x7f8028101870 2024-04-25T01:43:30.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.802+0000 7f8030231640 1 -- 172.21.15.78:0/3225161457 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80281a0070 con 0x7f8028101870 2024-04-25T01:43:30.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.803+0000 7f8030231640 1 -- 172.21.15.78:0/3225161457 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8028107a70 con 0x7f8028101870 2024-04-25T01:43:30.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.804+0000 7f801effd640 1 -- 172.21.15.78:0/3225161457 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8018017070 con 0x7f8028101870 2024-04-25T01:43:30.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.804+0000 7f801effd640 1 --2- 172.21.15.78:0/3225161457 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f80140a2b90 0x7f80140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:30.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.805+0000 7f802dfcd640 1 --2- 172.21.15.78:0/3225161457 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f80140a2b90 0x7f80140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:30.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.805+0000 7f801effd640 1 -- 172.21.15.78:0/3225161457 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f80180ce6a0 con 0x7f8028101870 2024-04-25T01:43:30.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.805+0000 7f802dfcd640 1 --2- 172.21.15.78:0/3225161457 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f80140a2b90 0x7f80140a5050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8020006010 tx=0x7f8020000f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:30.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.809+0000 7f801effd640 1 -- 172.21.15.78:0/3225161457 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f801809b3c0 con 0x7f8028101870 2024-04-25T01:43:30.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.987+0000 7f8030231640 1 -- 172.21.15.78:0/3225161457 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8028062340 con 0x7f8028101870 2024-04-25T01:43:30.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.988+0000 7f801effd640 1 -- 172.21.15.78:0/3225161457 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24151 (secure 0 0 0) 0x7f801809acc0 con 0x7f8028101870 2024-04-25T01:43:30.979 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:30.979 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:43:21.781660+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:43:30.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.990+0000 7f8030231640 1 -- 172.21.15.78:0/3225161457 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f80140a2b90 msgr2=0x7f80140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.990+0000 7f8030231640 1 --2- 172.21.15.78:0/3225161457 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f80140a2b90 0x7f80140a5050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8020006010 tx=0x7f8020000f40 comp rx=0 tx=0).stop 2024-04-25T01:43:30.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.990+0000 7f8030231640 1 -- 172.21.15.78:0/3225161457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8028101870 msgr2=0x7f802819c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:30.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.990+0000 7f8030231640 1 --2- 172.21.15.78:0/3225161457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8028101870 0x7f802819c930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f801800b4a0 tx=0x7f801800b960 comp rx=0 tx=0).stop 2024-04-25T01:43:30.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.990+0000 7f8030231640 1 -- 172.21.15.78:0/3225161457 shutdown_connections 2024-04-25T01:43:30.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.990+0000 7f8030231640 1 --2- 172.21.15.78:0/3225161457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8028102230 0x7f802819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.990+0000 7f8030231640 1 --2- 172.21.15.78:0/3225161457 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f80140a2b90 0x7f80140a5050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.990+0000 7f8030231640 1 --2- 172.21.15.78:0/3225161457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8028101870 0x7f802819c930 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.990+0000 7f8030231640 1 --2- 172.21.15.78:0/3225161457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8028100680 0x7f8028069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:30.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.990+0000 7f8030231640 1 -- 172.21.15.78:0/3225161457 >> 172.21.15.78:0/3225161457 conn(0x7f80280fbbf0 msgr2=0x7f8028104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:30.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.990+0000 7f8030231640 1 -- 172.21.15.78:0/3225161457 shutdown_connections 2024-04-25T01:43:30.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:30.991+0000 7f8030231640 1 -- 172.21.15.78:0/3225161457 wait complete. 2024-04-25T01:43:30.992 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:38:39.136418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:43:15.613881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:43:16.600270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:43:19.650457+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:43:30.993 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:43:31.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.102+0000 7f8883114640 1 -- 172.21.15.78:0/3534894128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f887c101860 msgr2=0x7f887c101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.102+0000 7f8883114640 1 --2- 172.21.15.78:0/3534894128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f887c101860 0x7f887c101ce0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8870009900 tx=0x7f887002a410 comp rx=0 tx=0).stop 2024-04-25T01:43:31.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.102+0000 7f8883114640 1 -- 172.21.15.78:0/3534894128 shutdown_connections 2024-04-25T01:43:31.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.102+0000 7f8883114640 1 --2- 172.21.15.78:0/3534894128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f887c102220 0x7f887c108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.102+0000 7f8883114640 1 --2- 172.21.15.78:0/3534894128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f887c101860 0x7f887c101ce0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.102+0000 7f8883114640 1 --2- 172.21.15.78:0/3534894128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f887c100670 0x7f887c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.102+0000 7f8883114640 1 -- 172.21.15.78:0/3534894128 >> 172.21.15.78:0/3534894128 conn(0x7f887c0fbbe0 msgr2=0x7f887c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:31.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.102+0000 7f8883114640 1 -- 172.21.15.78:0/3534894128 shutdown_connections 2024-04-25T01:43:31.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.102+0000 7f8883114640 1 -- 172.21.15.78:0/3534894128 wait complete. 2024-04-25T01:43:31.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.102+0000 7f8883114640 1 Processor -- start 2024-04-25T01:43:31.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.102+0000 7f8883114640 1 -- start start 2024-04-25T01:43:31.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f8883114640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f887c100670 0x7f887c06a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f8883114640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f887c101860 0x7f887c19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f8883114640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f887c102220 0x7f887c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f8883114640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f887c10c300 con 0x7f887c102220 2024-04-25T01:43:31.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f8883114640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f887c10c180 con 0x7f887c101860 2024-04-25T01:43:31.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f8883114640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f887c10c480 con 0x7f887c100670 2024-04-25T01:43:31.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f88816b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f887c102220 0x7f887c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f88816b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f887c102220 0x7f887c19ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49284/0 (socket says 172.21.15.78:49284) 2024-04-25T01:43:31.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f88816b1640 1 -- 172.21.15.78:0/3104202996 learned_addr learned my addr 172.21.15.78:0/3104202996 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:31.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f887bfff640 1 --2- 172.21.15.78:0/3104202996 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f887c101860 0x7f887c19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f8880eb0640 1 --2- 172.21.15.78:0/3104202996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f887c100670 0x7f887c06a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f88816b1640 1 -- 172.21.15.78:0/3104202996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f887c100670 msgr2=0x7f887c06a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f88816b1640 1 --2- 172.21.15.78:0/3104202996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f887c100670 0x7f887c06a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f88816b1640 1 -- 172.21.15.78:0/3104202996 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f887c101860 msgr2=0x7f887c19c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f88816b1640 1 --2- 172.21.15.78:0/3104202996 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f887c101860 0x7f887c19c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f88816b1640 1 -- 172.21.15.78:0/3104202996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887c19f6e0 con 0x7f887c102220 2024-04-25T01:43:31.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f88816b1640 1 --2- 172.21.15.78:0/3104202996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f887c102220 0x7f887c19ee20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8874009fd0 tx=0x7f887400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:31.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f8879ffb640 1 -- 172.21.15.78:0/3104202996 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874017070 con 0x7f887c102220 2024-04-25T01:43:31.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f8879ffb640 1 -- 172.21.15.78:0/3104202996 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8874003e50 con 0x7f887c102220 2024-04-25T01:43:31.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f8883114640 1 -- 172.21.15.78:0/3104202996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f887c19f9d0 con 0x7f887c102220 2024-04-25T01:43:31.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.103+0000 7f8879ffb640 1 -- 172.21.15.78:0/3104202996 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88740106c0 con 0x7f887c102220 2024-04-25T01:43:31.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.104+0000 7f8883114640 1 -- 172.21.15.78:0/3104202996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f887c19fea0 con 0x7f887c102220 2024-04-25T01:43:31.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.104+0000 7f8883114640 1 -- 172.21.15.78:0/3104202996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f887c107a50 con 0x7f887c102220 2024-04-25T01:43:31.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.105+0000 7f8879ffb640 1 -- 172.21.15.78:0/3104202996 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8874003970 con 0x7f887c102220 2024-04-25T01:43:31.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.106+0000 7f8879ffb640 1 --2- 172.21.15.78:0/3104202996 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f88640a2b90 0x7f88640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.106+0000 7f8879ffb640 1 -- 172.21.15.78:0/3104202996 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f88740cf020 con 0x7f887c102220 2024-04-25T01:43:31.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.106+0000 7f8880eb0640 1 --2- 172.21.15.78:0/3104202996 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f88640a2b90 0x7f88640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.106+0000 7f8880eb0640 1 --2- 172.21.15.78:0/3104202996 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f88640a2b90 0x7f88640a5050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f886c005fd0 tx=0x7f886c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:31.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.110+0000 7f8879ffb640 1 -- 172.21.15.78:0/3104202996 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f887409bd40 con 0x7f887c102220 2024-04-25T01:43:31.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:30 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/153939415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:43:31.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:30 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/968127596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:43:31.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:30 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/501757765' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T01:43:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.286+0000 7f8883114640 1 -- 172.21.15.78:0/3104202996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f887c062340 con 0x7f887c102220 2024-04-25T01:43:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.286+0000 7f8879ffb640 1 -- 172.21.15.78:0/3104202996 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f8874097020 con 0x7f887c102220 2024-04-25T01:43:31.277 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T01:43:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.288+0000 7f8883114640 1 -- 172.21.15.78:0/3104202996 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f88640a2b90 msgr2=0x7f88640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.288+0000 7f8883114640 1 --2- 172.21.15.78:0/3104202996 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f88640a2b90 0x7f88640a5050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f886c005fd0 tx=0x7f886c005850 comp rx=0 tx=0).stop 2024-04-25T01:43:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.288+0000 7f8883114640 1 -- 172.21.15.78:0/3104202996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f887c102220 msgr2=0x7f887c19ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.288+0000 7f8883114640 1 --2- 172.21.15.78:0/3104202996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f887c102220 0x7f887c19ee20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8874009fd0 tx=0x7f887400b260 comp rx=0 tx=0).stop 2024-04-25T01:43:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.288+0000 7f8883114640 1 -- 172.21.15.78:0/3104202996 shutdown_connections 2024-04-25T01:43:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.288+0000 7f8883114640 1 --2- 172.21.15.78:0/3104202996 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f88640a2b90 0x7f88640a5050 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.288+0000 7f8883114640 1 --2- 172.21.15.78:0/3104202996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f887c102220 0x7f887c19ee20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.288+0000 7f8883114640 1 --2- 172.21.15.78:0/3104202996 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f887c101860 0x7f887c19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.288+0000 7f8883114640 1 --2- 172.21.15.78:0/3104202996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f887c100670 0x7f887c06a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.288+0000 7f8883114640 1 -- 172.21.15.78:0/3104202996 >> 172.21.15.78:0/3104202996 conn(0x7f887c0fbbe0 msgr2=0x7f887c104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.288+0000 7f8883114640 1 -- 172.21.15.78:0/3104202996 shutdown_connections 2024-04-25T01:43:31.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.288+0000 7f8883114640 1 -- 172.21.15.78:0/3104202996 wait complete. 2024-04-25T01:43:31.289 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:43:31.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:30 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/153939415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:43:31.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:30 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/968127596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:43:31.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:30 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/501757765' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T01:43:31.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:30 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/153939415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:43:31.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:30 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/968127596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:43:31.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:30 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/501757765' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T01:43:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.399+0000 7fe516412640 1 -- 172.21.15.78:0/1697633687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe510103350 msgr2=0x7fe510109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.399+0000 7fe516412640 1 --2- 172.21.15.78:0/1697633687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe510103350 0x7fe510109be0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe508009ca0 tx=0x7fe50802a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.399+0000 7fe516412640 1 -- 172.21.15.78:0/1697633687 shutdown_connections 2024-04-25T01:43:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.399+0000 7fe516412640 1 --2- 172.21.15.78:0/1697633687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe510103350 0x7fe510109be0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.399+0000 7fe516412640 1 --2- 172.21.15.78:0/1697633687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe510102990 0x7fe510102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.399+0000 7fe516412640 1 --2- 172.21.15.78:0/1697633687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5101017a0 0x7fe510101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.399+0000 7fe516412640 1 -- 172.21.15.78:0/1697633687 >> 172.21.15.78:0/1697633687 conn(0x7fe5100fcd10 msgr2=0x7fe5100ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.400+0000 7fe516412640 1 -- 172.21.15.78:0/1697633687 shutdown_connections 2024-04-25T01:43:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.400+0000 7fe516412640 1 -- 172.21.15.78:0/1697633687 wait complete. 2024-04-25T01:43:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.400+0000 7fe516412640 1 Processor -- start 2024-04-25T01:43:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.400+0000 7fe516412640 1 -- start start 2024-04-25T01:43:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.400+0000 7fe516412640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5101017a0 0x7fe5101979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.400+0000 7fe516412640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510102990 0x7fe510197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.400+0000 7fe516412640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe510103350 0x7fe51019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.400+0000 7fe516412640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe51010d400 con 0x7fe510103350 2024-04-25T01:43:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.400+0000 7fe516412640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe51010d280 con 0x7fe510102990 2024-04-25T01:43:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.400+0000 7fe516412640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe51010d580 con 0x7fe5101017a0 2024-04-25T01:43:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.400+0000 7fe5149af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe510103350 0x7fe51019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe5149af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe510103350 0x7fe51019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49298/0 (socket says 172.21.15.78:49298) 2024-04-25T01:43:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe5149af640 1 -- 172.21.15.78:0/1076201260 learned_addr learned my addr 172.21.15.78:0/1076201260 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe50f7fe640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510102990 0x7fe510197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe50ffff640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5101017a0 0x7fe5101979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe5149af640 1 -- 172.21.15.78:0/1076201260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5101017a0 msgr2=0x7fe5101979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe5149af640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5101017a0 0x7fe5101979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe5149af640 1 -- 172.21.15.78:0/1076201260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510102990 msgr2=0x7fe510197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe5149af640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510102990 0x7fe510197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe5149af640 1 -- 172.21.15.78:0/1076201260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51019f5e0 con 0x7fe510103350 2024-04-25T01:43:31.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe50f7fe640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510102990 0x7fe510197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:43:31.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe5149af640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe510103350 0x7fe51019ef70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe508031ae0 tx=0x7fe508031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:31.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe50d7fa640 1 -- 172.21.15.78:0/1076201260 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe508003b70 con 0x7fe510103350 2024-04-25T01:43:31.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe516412640 1 -- 172.21.15.78:0/1076201260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51019f870 con 0x7fe510103350 2024-04-25T01:43:31.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe50d7fa640 1 -- 172.21.15.78:0/1076201260 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe508003d10 con 0x7fe510103350 2024-04-25T01:43:31.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe50d7fa640 1 -- 172.21.15.78:0/1076201260 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50802fc20 con 0x7fe510103350 2024-04-25T01:43:31.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.401+0000 7fe516412640 1 -- 172.21.15.78:0/1076201260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51019fe10 con 0x7fe510103350 2024-04-25T01:43:31.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.402+0000 7fe516412640 1 -- 172.21.15.78:0/1076201260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe510108a60 con 0x7fe510103350 2024-04-25T01:43:31.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.403+0000 7fe50d7fa640 1 -- 172.21.15.78:0/1076201260 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe5080046c0 con 0x7fe510103350 2024-04-25T01:43:31.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.404+0000 7fe50d7fa640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4fc0a2b90 0x7fe4fc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.404+0000 7fe50ffff640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4fc0a2b90 0x7fe4fc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.404+0000 7fe50d7fa640 1 -- 172.21.15.78:0/1076201260 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe5080f0370 con 0x7fe510103350 2024-04-25T01:43:31.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.404+0000 7fe50ffff640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4fc0a2b90 0x7fe4fc0a5050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe5101027f0 tx=0x7fe4f8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:31.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.408+0000 7fe50d7fa640 1 -- 172.21.15.78:0/1076201260 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe5080bd1b0 con 0x7fe510103350 2024-04-25T01:43:31.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.581+0000 7fe516412640 1 -- 172.21.15.78:0/1076201260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe510062340 con 0x7fe510103350 2024-04-25T01:43:31.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.582+0000 7fe50d7fa640 1 -- 172.21.15.78:0/1076201260 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fe5080bcab0 con 0x7fe510103350 2024-04-25T01:43:31.573 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:43:31.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.584+0000 7fe516412640 1 -- 172.21.15.78:0/1076201260 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4fc0a2b90 msgr2=0x7fe4fc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.584+0000 7fe516412640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4fc0a2b90 0x7fe4fc0a5050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe5101027f0 tx=0x7fe4f8005f90 comp rx=0 tx=0).stop 2024-04-25T01:43:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.584+0000 7fe516412640 1 -- 172.21.15.78:0/1076201260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe510103350 msgr2=0x7fe51019ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.584+0000 7fe516412640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe510103350 0x7fe51019ef70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe508031ae0 tx=0x7fe508031ee0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.584+0000 7fe516412640 1 -- 172.21.15.78:0/1076201260 shutdown_connections 2024-04-25T01:43:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.584+0000 7fe516412640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4fc0a2b90 0x7fe4fc0a5050 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.584+0000 7fe516412640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe510103350 0x7fe51019ef70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.584+0000 7fe516412640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe510102990 0x7fe510197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.584+0000 7fe516412640 1 --2- 172.21.15.78:0/1076201260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5101017a0 0x7fe5101979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.584+0000 7fe516412640 1 -- 172.21.15.78:0/1076201260 >> 172.21.15.78:0/1076201260 conn(0x7fe5100fcd10 msgr2=0x7fe510105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.584+0000 7fe516412640 1 -- 172.21.15.78:0/1076201260 shutdown_connections 2024-04-25T01:43:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.584+0000 7fe516412640 1 -- 172.21.15.78:0/1076201260 wait complete. 2024-04-25T01:43:31.585 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:43:31.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.695+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/3529310918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e44105310 msgr2=0x7f2e44105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.695+0000 7f2e4bfaf640 1 --2- 172.21.15.78:0/3529310918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e44105310 0x7f2e44105790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009ca0 tx=0x7f2e4002a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.695+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/3529310918 shutdown_connections 2024-04-25T01:43:31.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.695+0000 7f2e4bfaf640 1 --2- 172.21.15.78:0/3529310918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e44105310 0x7f2e44105790 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.695+0000 7f2e4bfaf640 1 --2- 172.21.15.78:0/3529310918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e44102960 0x7f2e44104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.695+0000 7f2e4bfaf640 1 --2- 172.21.15.78:0/3529310918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e440ffff0 0x7f2e44102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.695+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/3529310918 >> 172.21.15.78:0/3529310918 conn(0x7f2e440fbc30 msgr2=0x7f2e440fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:31.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.695+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/3529310918 shutdown_connections 2024-04-25T01:43:31.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.695+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/3529310918 wait complete. 2024-04-25T01:43:31.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.695+0000 7f2e4bfaf640 1 Processor -- start 2024-04-25T01:43:31.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.695+0000 7f2e4bfaf640 1 -- start start 2024-04-25T01:43:31.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4bfaf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e440ffff0 0x7f2e4419e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4bfaf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e44102960 0x7f2e4419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4bfaf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e44105310 0x7f2e441a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4bfaf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441139f0 con 0x7f2e44105310 2024-04-25T01:43:31.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4bfaf640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44113870 con 0x7f2e44102960 2024-04-25T01:43:31.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4bfaf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44113b70 con 0x7f2e440ffff0 2024-04-25T01:43:31.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4a54c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e44105310 0x7f2e441a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4a54c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e44105310 0x7f2e441a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49310/0 (socket says 172.21.15.78:49310) 2024-04-25T01:43:31.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4a54c640 1 -- 172.21.15.78:0/21861290 learned_addr learned my addr 172.21.15.78:0/21861290 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:31.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e49d4b640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e440ffff0 0x7f2e4419e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4954a640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e44102960 0x7f2e4419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4a54c640 1 -- 172.21.15.78:0/21861290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e440ffff0 msgr2=0x7f2e4419e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4a54c640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e440ffff0 0x7f2e4419e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4a54c640 1 -- 172.21.15.78:0/21861290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e44102960 msgr2=0x7f2e4419e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4a54c640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e44102960 0x7f2e4419e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4a54c640 1 -- 172.21.15.78:0/21861290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e441a5c70 con 0x7f2e44105310 2024-04-25T01:43:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4954a640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e44102960 0x7f2e4419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4a54c640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e44105310 0x7f2e441a5600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2e40031ae0 tx=0x7f2e40031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e3affd640 1 -- 172.21.15.78:0/21861290 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e40003b70 con 0x7f2e44105310 2024-04-25T01:43:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e3affd640 1 -- 172.21.15.78:0/21861290 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e40003d10 con 0x7f2e44105310 2024-04-25T01:43:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e3affd640 1 -- 172.21.15.78:0/21861290 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4002fc40 con 0x7f2e44105310 2024-04-25T01:43:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/21861290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e441a5f00 con 0x7f2e44105310 2024-04-25T01:43:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.696+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/21861290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441a63c0 con 0x7f2e44105310 2024-04-25T01:43:31.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.697+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/21861290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e440fd720 con 0x7f2e44105310 2024-04-25T01:43:31.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.698+0000 7f2e3affd640 1 -- 172.21.15.78:0/21861290 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2e40004050 con 0x7f2e44105310 2024-04-25T01:43:31.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.699+0000 7f2e3affd640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2e300a2b90 0x7f2e300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.699+0000 7f2e49d4b640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2e300a2b90 0x7f2e300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.699+0000 7f2e3affd640 1 -- 172.21.15.78:0/21861290 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2e400f0560 con 0x7f2e44105310 2024-04-25T01:43:31.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.699+0000 7f2e49d4b640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2e300a2b90 0x7f2e300a5050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009bd0 tx=0x7f2e3c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:31.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.703+0000 7f2e3affd640 1 -- 172.21.15.78:0/21861290 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2e400bd280 con 0x7f2e44105310 2024-04-25T01:43:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.880+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/21861290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2e440623e0 con 0x7f2e44105310 2024-04-25T01:43:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.881+0000 7f2e3affd640 1 -- 172.21.15.78:0/21861290 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f2e400bcb80 con 0x7f2e44105310 2024-04-25T01:43:31.872 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T01:43:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.883+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/21861290 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2e300a2b90 msgr2=0x7f2e300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.883+0000 7f2e4bfaf640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2e300a2b90 0x7f2e300a5050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009bd0 tx=0x7f2e3c005f90 comp rx=0 tx=0).stop 2024-04-25T01:43:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.883+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/21861290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e44105310 msgr2=0x7f2e441a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.883+0000 7f2e4bfaf640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e44105310 0x7f2e441a5600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2e40031ae0 tx=0x7f2e40031ee0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.883+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/21861290 shutdown_connections 2024-04-25T01:43:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.883+0000 7f2e4bfaf640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2e300a2b90 0x7f2e300a5050 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.883+0000 7f2e4bfaf640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e44105310 0x7f2e441a5600 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.883+0000 7f2e4bfaf640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e44102960 0x7f2e4419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.883+0000 7f2e4bfaf640 1 --2- 172.21.15.78:0/21861290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e440ffff0 0x7f2e4419e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.883+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/21861290 >> 172.21.15.78:0/21861290 conn(0x7f2e440fbc30 msgr2=0x7f2e44100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.883+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/21861290 shutdown_connections 2024-04-25T01:43:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.883+0000 7f2e4bfaf640 1 -- 172.21.15.78:0/21861290 wait complete. 2024-04-25T01:43:31.884 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:43:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.997+0000 7f2677aa5640 1 -- 172.21.15.78:0/4124144192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2670102200 msgr2=0x7f26701045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.997+0000 7f2677aa5640 1 --2- 172.21.15.78:0/4124144192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2670102200 0x7f26701045f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f266c009ca0 tx=0x7f266c02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.997+0000 7f2677aa5640 1 -- 172.21.15.78:0/4124144192 shutdown_connections 2024-04-25T01:43:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.997+0000 7f2677aa5640 1 --2- 172.21.15.78:0/4124144192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2670107460 0x7f26701098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.997+0000 7f2677aa5640 1 --2- 172.21.15.78:0/4124144192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2670104b30 0x7f2670106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.997+0000 7f2677aa5640 1 --2- 172.21.15.78:0/4124144192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2670102200 0x7f26701045f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.997+0000 7f2677aa5640 1 -- 172.21.15.78:0/4124144192 >> 172.21.15.78:0/4124144192 conn(0x7f26700fbbf0 msgr2=0x7f26700fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.997+0000 7f2677aa5640 1 -- 172.21.15.78:0/4124144192 shutdown_connections 2024-04-25T01:43:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.997+0000 7f2677aa5640 1 -- 172.21.15.78:0/4124144192 wait complete. 2024-04-25T01:43:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2677aa5640 1 Processor -- start 2024-04-25T01:43:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2677aa5640 1 -- start start 2024-04-25T01:43:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2677aa5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2670102200 0x7f2670197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2677aa5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2670104b30 0x7f2670198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2677aa5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2670107460 0x7f267019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2677aa5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f267010d400 con 0x7f2670102200 2024-04-25T01:43:31.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2677aa5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f267010d280 con 0x7f2670104b30 2024-04-25T01:43:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2677aa5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f267010d580 con 0x7f2670107460 2024-04-25T01:43:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2675841640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2670102200 0x7f2670197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2675841640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2670102200 0x7f2670197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49318/0 (socket says 172.21.15.78:49318) 2024-04-25T01:43:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2675841640 1 -- 172.21.15.78:0/1949208873 learned_addr learned my addr 172.21.15.78:0/1949208873 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2675040640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2670104b30 0x7f2670198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2676042640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2670107460 0x7f267019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2675841640 1 -- 172.21.15.78:0/1949208873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2670107460 msgr2=0x7f267019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2675841640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2670107460 0x7f267019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2675841640 1 -- 172.21.15.78:0/1949208873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2670104b30 msgr2=0x7f2670198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2675841640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2670104b30 0x7f2670198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.998+0000 7f2675841640 1 -- 172.21.15.78:0/1949208873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267019f720 con 0x7f2670102200 2024-04-25T01:43:31.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.999+0000 7f2676042640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2670107460 0x7f267019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.999+0000 7f2675841640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2670102200 0x7f2670197b00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f266c031a40 tx=0x7f266c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.999+0000 7f2666ffd640 1 -- 172.21.15.78:0/1949208873 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c02fa20 con 0x7f2670102200 2024-04-25T01:43:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.999+0000 7f2677aa5640 1 -- 172.21.15.78:0/1949208873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267019f9b0 con 0x7f2670102200 2024-04-25T01:43:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.999+0000 7f2666ffd640 1 -- 172.21.15.78:0/1949208873 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f266c02fbc0 con 0x7f2670102200 2024-04-25T01:43:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.999+0000 7f2666ffd640 1 -- 172.21.15.78:0/1949208873 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c03a780 con 0x7f2670102200 2024-04-25T01:43:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:31.999+0000 7f2677aa5640 1 -- 172.21.15.78:0/1949208873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267019ff00 con 0x7f2670102200 2024-04-25T01:43:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.000+0000 7f2677aa5640 1 -- 172.21.15.78:0/1949208873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26700fd6e0 con 0x7f2670102200 2024-04-25T01:43:31.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.001+0000 7f2666ffd640 1 -- 172.21.15.78:0/1949208873 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f266c03a920 con 0x7f2670102200 2024-04-25T01:43:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.001+0000 7f2666ffd640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f265c0a2b90 0x7f265c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.001+0000 7f2675040640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f265c0a2b90 0x7f265c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.001+0000 7f2666ffd640 1 -- 172.21.15.78:0/1949208873 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f266c0f08b0 con 0x7f2670102200 2024-04-25T01:43:31.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.002+0000 7f2675040640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f265c0a2b90 0x7f265c0a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2670199010 tx=0x7f266000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:31.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.006+0000 7f2666ffd640 1 -- 172.21.15.78:0/1949208873 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f266c0bd5d0 con 0x7f2670102200 2024-04-25T01:43:32.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.180+0000 7f2677aa5640 1 -- 172.21.15.78:0/1949208873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2670062340 con 0x7f2670102200 2024-04-25T01:43:32.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.180+0000 7f2666ffd640 1 -- 172.21.15.78:0/1949208873 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f266c0bced0 con 0x7f2670102200 2024-04-25T01:43:32.171 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:43:32.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.182+0000 7f2677aa5640 1 -- 172.21.15.78:0/1949208873 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f265c0a2b90 msgr2=0x7f265c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:32.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.182+0000 7f2677aa5640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f265c0a2b90 0x7f265c0a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2670199010 tx=0x7f266000b410 comp rx=0 tx=0).stop 2024-04-25T01:43:32.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.182+0000 7f2677aa5640 1 -- 172.21.15.78:0/1949208873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2670102200 msgr2=0x7f2670197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:32.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.182+0000 7f2677aa5640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2670102200 0x7f2670197b00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f266c031a40 tx=0x7f266c031ee0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.182+0000 7f2677aa5640 1 -- 172.21.15.78:0/1949208873 shutdown_connections 2024-04-25T01:43:32.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.182+0000 7f2677aa5640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2670107460 0x7f267019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.182+0000 7f2677aa5640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2670104b30 0x7f2670198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.182+0000 7f2677aa5640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f265c0a2b90 0x7f265c0a5050 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.182+0000 7f2677aa5640 1 --2- 172.21.15.78:0/1949208873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2670102200 0x7f2670197b00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.182+0000 7f2677aa5640 1 -- 172.21.15.78:0/1949208873 >> 172.21.15.78:0/1949208873 conn(0x7f26700fbbf0 msgr2=0x7f2670104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:32.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.182+0000 7f2677aa5640 1 -- 172.21.15.78:0/1949208873 shutdown_connections 2024-04-25T01:43:32.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.182+0000 7f2677aa5640 1 -- 172.21.15.78:0/1949208873 wait complete. 2024-04-25T01:43:32.183 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:31 smithi078 ceph-mon[29469]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 432 B/s wr, 1 op/s 2024-04-25T01:43:32.183 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:31 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3225161457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:32.183 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:31 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3104202996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:43:32.183 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:31 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1076201260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:43:32.183 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:31 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/21861290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:43:32.184 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:43:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.291+0000 7f482c712640 1 -- 172.21.15.78:0/1430061243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4824102230 msgr2=0x7f4824108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.291+0000 7f482c712640 1 --2- 172.21.15.78:0/1430061243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4824102230 0x7f4824108ac0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f481c009900 tx=0x7f481c02a410 comp rx=0 tx=0).stop 2024-04-25T01:43:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.291+0000 7f482c712640 1 -- 172.21.15.78:0/1430061243 shutdown_connections 2024-04-25T01:43:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.291+0000 7f482c712640 1 --2- 172.21.15.78:0/1430061243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4824102230 0x7f4824108ac0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.291+0000 7f482c712640 1 --2- 172.21.15.78:0/1430061243 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4824101870 0x7f4824101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.291+0000 7f482c712640 1 --2- 172.21.15.78:0/1430061243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4824100680 0x7f4824100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.291+0000 7f482c712640 1 -- 172.21.15.78:0/1430061243 >> 172.21.15.78:0/1430061243 conn(0x7f48240fbbf0 msgr2=0x7f48240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.291+0000 7f482c712640 1 -- 172.21.15.78:0/1430061243 shutdown_connections 2024-04-25T01:43:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.291+0000 7f482c712640 1 -- 172.21.15.78:0/1430061243 wait complete. 2024-04-25T01:43:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.292+0000 7f482c712640 1 Processor -- start 2024-04-25T01:43:32.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.292+0000 7f482c712640 1 -- start start 2024-04-25T01:43:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.292+0000 7f482c712640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4824100680 0x7f4824069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.292+0000 7f482c712640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4824101870 0x7f482419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.292+0000 7f482c712640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4824102230 0x7f482419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.292+0000 7f482c712640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f482410c300 con 0x7f4824100680 2024-04-25T01:43:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.292+0000 7f482c712640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f482410c180 con 0x7f4824101870 2024-04-25T01:43:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.292+0000 7f482c712640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f482410c480 con 0x7f4824102230 2024-04-25T01:43:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.292+0000 7f482a4ae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4824100680 0x7f4824069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.292+0000 7f482a4ae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4824100680 0x7f4824069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49336/0 (socket says 172.21.15.78:49336) 2024-04-25T01:43:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.292+0000 7f482a4ae640 1 -- 172.21.15.78:0/3789093308 learned_addr learned my addr 172.21.15.78:0/3789093308 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.292+0000 7f482acaf640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4824102230 0x7f482419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.292+0000 7f4829cad640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4824101870 0x7f482419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:32.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.293+0000 7f482a4ae640 1 -- 172.21.15.78:0/3789093308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4824102230 msgr2=0x7f482419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.293+0000 7f482a4ae640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4824102230 0x7f482419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.293+0000 7f482a4ae640 1 -- 172.21.15.78:0/3789093308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4824101870 msgr2=0x7f482419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.293+0000 7f482a4ae640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4824101870 0x7f482419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.293+0000 7f482a4ae640 1 -- 172.21.15.78:0/3789093308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482419f6f0 con 0x7f4824100680 2024-04-25T01:43:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.293+0000 7f482acaf640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4824102230 0x7f482419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:43:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.293+0000 7f482a4ae640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4824100680 0x7f4824069440 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f482000eb70 tx=0x7f482000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.293+0000 7f481b7fe640 1 -- 172.21.15.78:0/3789093308 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820003b70 con 0x7f4824100680 2024-04-25T01:43:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.293+0000 7f481b7fe640 1 -- 172.21.15.78:0/3789093308 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f482000ee70 con 0x7f4824100680 2024-04-25T01:43:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.293+0000 7f482c712640 1 -- 172.21.15.78:0/3789093308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482419f980 con 0x7f4824100680 2024-04-25T01:43:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.293+0000 7f481b7fe640 1 -- 172.21.15.78:0/3789093308 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820010830 con 0x7f4824100680 2024-04-25T01:43:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.293+0000 7f482c712640 1 -- 172.21.15.78:0/3789093308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f482419fe80 con 0x7f4824100680 2024-04-25T01:43:32.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.294+0000 7f482c712640 1 -- 172.21.15.78:0/3789093308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48241079a0 con 0x7f4824100680 2024-04-25T01:43:32.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.295+0000 7f481b7fe640 1 -- 172.21.15.78:0/3789093308 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4820004020 con 0x7f4824100680 2024-04-25T01:43:32.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.295+0000 7f481b7fe640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f48100a2b90 0x7f48100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:32.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.296+0000 7f4829cad640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f48100a2b90 0x7f48100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:32.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.296+0000 7f481b7fe640 1 -- 172.21.15.78:0/3789093308 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f48200cf910 con 0x7f4824100680 2024-04-25T01:43:32.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.296+0000 7f4829cad640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f48100a2b90 0x7f48100a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f48240691b0 tx=0x7f481400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:32.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.300+0000 7f481b7fe640 1 -- 172.21.15.78:0/3789093308 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f482009c630 con 0x7f4824100680 2024-04-25T01:43:32.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:31 smithi152 ceph-mon[32465]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 432 B/s wr, 1 op/s 2024-04-25T01:43:32.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:31 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3225161457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:32.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:31 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3104202996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:43:32.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:31 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1076201260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:43:32.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:31 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/21861290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:43:32.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:31 smithi196 ceph-mon[32978]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 432 B/s wr, 1 op/s 2024-04-25T01:43:32.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:31 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3225161457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:32.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:31 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3104202996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:43:32.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:31 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1076201260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:43:32.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:31 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/21861290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:43:32.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.487+0000 7f482c712640 1 -- 172.21.15.78:0/3789093308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4824062340 con 0x7f4824100680 2024-04-25T01:43:32.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.487+0000 7f481b7fe640 1 -- 172.21.15.78:0/3789093308 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+589 (secure 0 0 0) 0x7f482009bf30 con 0x7f4824100680 2024-04-25T01:43:32.478 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1101495738 2024-04-26T01:36:16.812502+0000 2024-04-25T01:43:32.478 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1101495738 2024-04-26T01:36:16.812502+0000 2024-04-25T01:43:32.479 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2762765405 2024-04-26T01:36:16.812502+0000 2024-04-25T01:43:32.479 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/4170716207 2024-04-26T01:35:50.507447+0000 2024-04-25T01:43:32.479 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1136116795 2024-04-26T01:35:50.507447+0000 2024-04-25T01:43:32.479 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1544860340 2024-04-26T01:35:50.507447+0000 2024-04-25T01:43:32.479 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1544860340 2024-04-26T01:35:50.507447+0000 2024-04-25T01:43:32.479 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/211357528 2024-04-26T01:36:16.812502+0000 2024-04-25T01:43:32.479 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/88994848 2024-04-26T01:36:16.812502+0000 2024-04-25T01:43:32.479 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1196975031 2024-04-26T01:35:50.507447+0000 2024-04-25T01:43:32.479 INFO:teuthology.orchestra.run.smithi078.stderr:listed 10 entries 2024-04-25T01:43:32.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.490+0000 7f482c712640 1 -- 172.21.15.78:0/3789093308 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f48100a2b90 msgr2=0x7f48100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:32.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.490+0000 7f482c712640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f48100a2b90 0x7f48100a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f48240691b0 tx=0x7f481400b390 comp rx=0 tx=0).stop 2024-04-25T01:43:32.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.490+0000 7f482c712640 1 -- 172.21.15.78:0/3789093308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4824100680 msgr2=0x7f4824069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:32.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.490+0000 7f482c712640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4824100680 0x7f4824069440 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f482000eb70 tx=0x7f482000c6a0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.490+0000 7f482c712640 1 -- 172.21.15.78:0/3789093308 shutdown_connections 2024-04-25T01:43:32.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.490+0000 7f482c712640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4824102230 0x7f482419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.490+0000 7f482c712640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4824101870 0x7f482419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.490+0000 7f482c712640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f48100a2b90 0x7f48100a5050 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.490+0000 7f482c712640 1 --2- 172.21.15.78:0/3789093308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4824100680 0x7f4824069440 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:32.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.490+0000 7f482c712640 1 -- 172.21.15.78:0/3789093308 >> 172.21.15.78:0/3789093308 conn(0x7f48240fbbf0 msgr2=0x7f4824104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:32.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.490+0000 7f482c712640 1 -- 172.21.15.78:0/3789093308 shutdown_connections 2024-04-25T01:43:32.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:32.490+0000 7f482c712640 1 -- 172.21.15.78:0/3789093308 wait complete. 2024-04-25T01:43:32.491 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2024-04-25T01:43:32.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:43:32.491 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:43:32.514 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi078.stderr:ceph-fuse[70628]: fuse finished with error 0 and tester_r 0 2024-04-25T01:43:32.515 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:43:33.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:32 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1949208873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:43:33.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:32 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3789093308' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T01:43:33.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:32 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1949208873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:43:33.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:32 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3789093308' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T01:43:33.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:32 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1949208873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:43:33.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:32 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3789093308' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T01:43:34.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:33 smithi078 ceph-mon[29469]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 293 B/s wr, 0 op/s 2024-04-25T01:43:34.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:33 smithi152 ceph-mon[32465]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 293 B/s wr, 0 op/s 2024-04-25T01:43:34.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:33 smithi196 ceph-mon[32978]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 293 B/s wr, 0 op/s 2024-04-25T01:43:36.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:35 smithi078 ceph-mon[29469]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T01:43:36.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:35 smithi152 ceph-mon[32465]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T01:43:36.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:35 smithi196 ceph-mon[32978]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T01:43:38.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:37 smithi078 ceph-mon[29469]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T01:43:38.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:37 smithi152 ceph-mon[32465]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T01:43:38.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:37 smithi196 ceph-mon[32978]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T01:43:38.617 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T01:43:38.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:43:38.617 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T01:43:38.648 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:43:38.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.813+0000 7f281f529640 1 -- 172.21.15.78:0/1481095845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2818102960 msgr2=0x7f2818104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:38.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.813+0000 7f281f529640 1 --2- 172.21.15.78:0/1481095845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2818102960 0x7f2818104dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f280c009ca0 tx=0x7f280c02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:38.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.813+0000 7f281f529640 1 -- 172.21.15.78:0/1481095845 shutdown_connections 2024-04-25T01:43:38.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.813+0000 7f281f529640 1 --2- 172.21.15.78:0/1481095845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2818105310 0x7f2818105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:38.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.813+0000 7f281f529640 1 --2- 172.21.15.78:0/1481095845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2818102960 0x7f2818104dd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:38.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.813+0000 7f281f529640 1 --2- 172.21.15.78:0/1481095845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28180ffff0 0x7f2818102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:38.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.813+0000 7f281f529640 1 -- 172.21.15.78:0/1481095845 >> 172.21.15.78:0/1481095845 conn(0x7f28180fbbf0 msgr2=0x7f28180fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:38.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.813+0000 7f281f529640 1 -- 172.21.15.78:0/1481095845 shutdown_connections 2024-04-25T01:43:38.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.813+0000 7f281f529640 1 -- 172.21.15.78:0/1481095845 wait complete. 2024-04-25T01:43:38.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281f529640 1 Processor -- start 2024-04-25T01:43:38.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281f529640 1 -- start start 2024-04-25T01:43:38.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281f529640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28180ffff0 0x7f281819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:38.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281f529640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2818102960 0x7f281819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:38.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281f529640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2818105310 0x7f28181a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:38.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281f529640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28181139f0 con 0x7f2818102960 2024-04-25T01:43:38.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281f529640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2818113870 con 0x7f2818105310 2024-04-25T01:43:38.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281f529640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2818113b70 con 0x7f28180ffff0 2024-04-25T01:43:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281cac4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2818102960 0x7f281819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281cac4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2818102960 0x7f281819e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49354/0 (socket says 172.21.15.78:49354) 2024-04-25T01:43:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281cac4640 1 -- 172.21.15.78:0/1876283830 learned_addr learned my addr 172.21.15.78:0/1876283830 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281d2c5640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28180ffff0 0x7f281819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281dac6640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2818105310 0x7f28181a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281cac4640 1 -- 172.21.15.78:0/1876283830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28180ffff0 msgr2=0x7f281819e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281cac4640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28180ffff0 0x7f281819e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281cac4640 1 -- 172.21.15.78:0/1876283830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2818105310 msgr2=0x7f28181a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281cac4640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2818105310 0x7f28181a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281cac4640 1 -- 172.21.15.78:0/1876283830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28181a5c30 con 0x7f2818102960 2024-04-25T01:43:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.814+0000 7f281d2c5640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28180ffff0 0x7f281819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.815+0000 7f281cac4640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2818102960 0x7f281819e550 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f280c031a40 tx=0x7f280c031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:38.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.815+0000 7f28167fc640 1 -- 172.21.15.78:0/1876283830 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c02faa0 con 0x7f2818102960 2024-04-25T01:43:38.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.815+0000 7f28167fc640 1 -- 172.21.15.78:0/1876283830 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280c02fc40 con 0x7f2818102960 2024-04-25T01:43:38.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.815+0000 7f281f529640 1 -- 172.21.15.78:0/1876283830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28181a5ec0 con 0x7f2818102960 2024-04-25T01:43:38.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.815+0000 7f28167fc640 1 -- 172.21.15.78:0/1876283830 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c03a790 con 0x7f2818102960 2024-04-25T01:43:38.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.815+0000 7f281f529640 1 -- 172.21.15.78:0/1876283830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28181a6380 con 0x7f2818102960 2024-04-25T01:43:38.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.816+0000 7f281f529640 1 -- 172.21.15.78:0/1876283830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28180fd6e0 con 0x7f2818102960 2024-04-25T01:43:38.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.817+0000 7f28167fc640 1 -- 172.21.15.78:0/1876283830 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f280c030b30 con 0x7f2818102960 2024-04-25T01:43:38.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.818+0000 7f28167fc640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f28000a2b90 0x7f28000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:38.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.818+0000 7f281d2c5640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f28000a2b90 0x7f28000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:38.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.818+0000 7f28167fc640 1 -- 172.21.15.78:0/1876283830 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f280c0f0a20 con 0x7f2818102960 2024-04-25T01:43:38.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.818+0000 7f281d2c5640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f28000a2b90 0x7f28000a5050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f28080058c0 tx=0x7f2808005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:38.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:38.822+0000 7f28167fc640 1 -- 172.21.15.78:0/1876283830 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f280c0bd740 con 0x7f2818102960 2024-04-25T01:43:38.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.001+0000 7f281f529640 1 -- 172.21.15.78:0/1876283830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2818062340 con 0x7f2818102960 2024-04-25T01:43:38.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.002+0000 7f28167fc640 1 -- 172.21.15.78:0/1876283830 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24151 (secure 0 0 0) 0x7f280c0bd040 con 0x7f2818102960 2024-04-25T01:43:38.993 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:38.993 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:43:21.781660+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:43:38.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.004+0000 7f281f529640 1 -- 172.21.15.78:0/1876283830 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f28000a2b90 msgr2=0x7f28000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:38.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.004+0000 7f281f529640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f28000a2b90 0x7f28000a5050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f28080058c0 tx=0x7f2808005850 comp rx=0 tx=0).stop 2024-04-25T01:43:38.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.004+0000 7f281f529640 1 -- 172.21.15.78:0/1876283830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2818102960 msgr2=0x7f281819e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:38.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.004+0000 7f281f529640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2818102960 0x7f281819e550 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f280c031a40 tx=0x7f280c031ec0 comp rx=0 tx=0).stop 2024-04-25T01:43:38.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.004+0000 7f281f529640 1 -- 172.21.15.78:0/1876283830 shutdown_connections 2024-04-25T01:43:38.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.004+0000 7f281f529640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2818105310 0x7f28181a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:38.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.004+0000 7f281f529640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f28000a2b90 0x7f28000a5050 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:38.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.004+0000 7f281f529640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2818102960 0x7f281819e550 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:38.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.004+0000 7f281f529640 1 --2- 172.21.15.78:0/1876283830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28180ffff0 0x7f281819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:38.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.004+0000 7f281f529640 1 -- 172.21.15.78:0/1876283830 >> 172.21.15.78:0/1876283830 conn(0x7f28180fbbf0 msgr2=0x7f2818100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:38.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.005+0000 7f281f529640 1 -- 172.21.15.78:0/1876283830 shutdown_connections 2024-04-25T01:43:38.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.005+0000 7f281f529640 1 -- 172.21.15.78:0/1876283830 wait complete. 2024-04-25T01:43:39.007 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:43:39.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.113+0000 7f9baf73b640 1 -- 172.21.15.78:0/4257057926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ba8101890 msgr2=0x7f9ba8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:39.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.113+0000 7f9baf73b640 1 --2- 172.21.15.78:0/4257057926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ba8101890 0x7f9ba8101d10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009900 tx=0x7f9b9802a410 comp rx=0 tx=0).stop 2024-04-25T01:43:39.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.113+0000 7f9baf73b640 1 -- 172.21.15.78:0/4257057926 shutdown_connections 2024-04-25T01:43:39.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.113+0000 7f9baf73b640 1 --2- 172.21.15.78:0/4257057926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ba8102250 0x7f9ba8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.113+0000 7f9baf73b640 1 --2- 172.21.15.78:0/4257057926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ba8101890 0x7f9ba8101d10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.113+0000 7f9baf73b640 1 --2- 172.21.15.78:0/4257057926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ba81006a0 0x7f9ba8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.113+0000 7f9baf73b640 1 -- 172.21.15.78:0/4257057926 >> 172.21.15.78:0/4257057926 conn(0x7f9ba80fbc30 msgr2=0x7f9ba80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:39.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.113+0000 7f9baf73b640 1 -- 172.21.15.78:0/4257057926 shutdown_connections 2024-04-25T01:43:39.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.113+0000 7f9baf73b640 1 -- 172.21.15.78:0/4257057926 wait complete. 2024-04-25T01:43:39.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9baf73b640 1 Processor -- start 2024-04-25T01:43:39.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9baf73b640 1 -- start start 2024-04-25T01:43:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9baf73b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ba81006a0 0x7f9ba8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9baf73b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ba8101890 0x7f9ba8198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9baf73b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ba8102250 0x7f9ba819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9baf73b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba810c300 con 0x7f9ba8102250 2024-04-25T01:43:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9baf73b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba810c180 con 0x7f9ba8101890 2024-04-25T01:43:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9baf73b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba810c480 con 0x7f9ba81006a0 2024-04-25T01:43:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9badcd8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ba8102250 0x7f9ba819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9badcd8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ba8102250 0x7f9ba819f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49362/0 (socket says 172.21.15.78:49362) 2024-04-25T01:43:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9badcd8640 1 -- 172.21.15.78:0/19817090 learned_addr learned my addr 172.21.15.78:0/19817090 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9baccd6640 1 --2- 172.21.15.78:0/19817090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ba8101890 0x7f9ba8198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9bad4d7640 1 --2- 172.21.15.78:0/19817090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ba81006a0 0x7f9ba8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9badcd8640 1 -- 172.21.15.78:0/19817090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ba81006a0 msgr2=0x7f9ba8197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9badcd8640 1 --2- 172.21.15.78:0/19817090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ba81006a0 0x7f9ba8197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9badcd8640 1 -- 172.21.15.78:0/19817090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ba8101890 msgr2=0x7f9ba8198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:39.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.114+0000 7f9badcd8640 1 --2- 172.21.15.78:0/19817090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ba8101890 0x7f9ba8198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.115+0000 7f9badcd8640 1 -- 172.21.15.78:0/19817090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba819f710 con 0x7f9ba8102250 2024-04-25T01:43:39.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.115+0000 7f9badcd8640 1 --2- 172.21.15.78:0/19817090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ba8102250 0x7f9ba819f0a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400e970 tx=0x7f9ba400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:39.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.115+0000 7f9b9e7fc640 1 -- 172.21.15.78:0/19817090 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba400ccc0 con 0x7f9ba8102250 2024-04-25T01:43:39.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.115+0000 7f9b9e7fc640 1 -- 172.21.15.78:0/19817090 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ba400ce60 con 0x7f9ba8102250 2024-04-25T01:43:39.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.115+0000 7f9baf73b640 1 -- 172.21.15.78:0/19817090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba819fa00 con 0x7f9ba8102250 2024-04-25T01:43:39.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.115+0000 7f9b9e7fc640 1 -- 172.21.15.78:0/19817090 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4010490 con 0x7f9ba8102250 2024-04-25T01:43:39.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.115+0000 7f9baf73b640 1 -- 172.21.15.78:0/19817090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba819fed0 con 0x7f9ba8102250 2024-04-25T01:43:39.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.116+0000 7f9baf73b640 1 -- 172.21.15.78:0/19817090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba81a0160 con 0x7f9ba8102250 2024-04-25T01:43:39.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.117+0000 7f9b9e7fc640 1 -- 172.21.15.78:0/19817090 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9ba4005000 con 0x7f9ba8102250 2024-04-25T01:43:39.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.118+0000 7f9b9e7fc640 1 --2- 172.21.15.78:0/19817090 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f9b940a2b90 0x7f9b940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:39.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.118+0000 7f9bad4d7640 1 --2- 172.21.15.78:0/19817090 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f9b940a2b90 0x7f9b940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:39.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.118+0000 7f9b9e7fc640 1 -- 172.21.15.78:0/19817090 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9ba40cf9d0 con 0x7f9ba8102250 2024-04-25T01:43:39.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.118+0000 7f9bad4d7640 1 --2- 172.21.15.78:0/19817090 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f9b940a2b90 0x7f9b940a5050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0005c40 tx=0x7f9ba0005bd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:39.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.122+0000 7f9b9e7fc640 1 -- 172.21.15.78:0/19817090 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9ba4053740 con 0x7f9ba8102250 2024-04-25T01:43:39.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.299+0000 7f9baf73b640 1 -- 172.21.15.78:0/19817090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ba81a03d0 con 0x7f9ba8102250 2024-04-25T01:43:39.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.301+0000 7f9b9e7fc640 1 -- 172.21.15.78:0/19817090 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24151 (secure 0 0 0) 0x7f9ba409c690 con 0x7f9ba8102250 2024-04-25T01:43:39.292 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:39.292 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:43:21.781660+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:43:39.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.303+0000 7f9baf73b640 1 -- 172.21.15.78:0/19817090 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f9b940a2b90 msgr2=0x7f9b940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:39.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.303+0000 7f9baf73b640 1 --2- 172.21.15.78:0/19817090 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f9b940a2b90 0x7f9b940a5050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0005c40 tx=0x7f9ba0005bd0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.303+0000 7f9baf73b640 1 -- 172.21.15.78:0/19817090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ba8102250 msgr2=0x7f9ba819f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:39.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.303+0000 7f9baf73b640 1 --2- 172.21.15.78:0/19817090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ba8102250 0x7f9ba819f0a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400e970 tx=0x7f9ba400ed70 comp rx=0 tx=0).stop 2024-04-25T01:43:39.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.303+0000 7f9baf73b640 1 -- 172.21.15.78:0/19817090 shutdown_connections 2024-04-25T01:43:39.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.303+0000 7f9baf73b640 1 --2- 172.21.15.78:0/19817090 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f9b940a2b90 0x7f9b940a5050 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.303+0000 7f9baf73b640 1 --2- 172.21.15.78:0/19817090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ba8102250 0x7f9ba819f0a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.303+0000 7f9baf73b640 1 --2- 172.21.15.78:0/19817090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ba8101890 0x7f9ba8198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.303+0000 7f9baf73b640 1 --2- 172.21.15.78:0/19817090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ba81006a0 0x7f9ba8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.303+0000 7f9baf73b640 1 -- 172.21.15.78:0/19817090 >> 172.21.15.78:0/19817090 conn(0x7f9ba80fbc30 msgr2=0x7f9ba80fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:39.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.303+0000 7f9baf73b640 1 -- 172.21.15.78:0/19817090 shutdown_connections 2024-04-25T01:43:39.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.303+0000 7f9baf73b640 1 -- 172.21.15.78:0/19817090 wait complete. 2024-04-25T01:43:39.305 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T01:43:39.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.415+0000 7f36a5b34640 1 -- 172.21.15.78:0/3786727198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a0103340 msgr2=0x7f36a0109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:39.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.415+0000 7f36a5b34640 1 --2- 172.21.15.78:0/3786727198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a0103340 0x7f36a0109bd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3698009ca0 tx=0x7f369802a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.415+0000 7f36a5b34640 1 -- 172.21.15.78:0/3786727198 shutdown_connections 2024-04-25T01:43:39.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.415+0000 7f36a5b34640 1 --2- 172.21.15.78:0/3786727198 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a0103340 0x7f36a0109bd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.415+0000 7f36a5b34640 1 --2- 172.21.15.78:0/3786727198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36a0102980 0x7f36a0102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.415+0000 7f36a5b34640 1 --2- 172.21.15.78:0/3786727198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36a0101790 0x7f36a0101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.415+0000 7f36a5b34640 1 -- 172.21.15.78:0/3786727198 >> 172.21.15.78:0/3786727198 conn(0x7f36a00fcd20 msgr2=0x7f36a00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:39.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.415+0000 7f36a5b34640 1 -- 172.21.15.78:0/3786727198 shutdown_connections 2024-04-25T01:43:39.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.415+0000 7f36a5b34640 1 -- 172.21.15.78:0/3786727198 wait complete. 2024-04-25T01:43:39.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f36a5b34640 1 Processor -- start 2024-04-25T01:43:39.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f36a5b34640 1 -- start start 2024-04-25T01:43:39.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f36a5b34640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36a0101790 0x7f36a0197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:39.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f36a5b34640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36a0102980 0x7f36a0197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:39.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f36a5b34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a0103340 0x7f36a019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:39.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f36a5b34640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36a010d410 con 0x7f36a0103340 2024-04-25T01:43:39.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f36a5b34640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f36a010d290 con 0x7f36a0102980 2024-04-25T01:43:39.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f36a5b34640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f36a010d590 con 0x7f36a0101790 2024-04-25T01:43:39.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f369ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a0103340 0x7f36a019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:39.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f369ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a0103340 0x7f36a019f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49388/0 (socket says 172.21.15.78:49388) 2024-04-25T01:43:39.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f369ffff640 1 -- 172.21.15.78:0/1965765514 learned_addr learned my addr 172.21.15.78:0/1965765514 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:39.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f3697fff640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36a0102980 0x7f36a0197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:39.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f369f7fe640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36a0101790 0x7f36a0197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:39.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f369ffff640 1 -- 172.21.15.78:0/1965765514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36a0101790 msgr2=0x7f36a0197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:39.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f369ffff640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36a0101790 0x7f36a0197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f369ffff640 1 -- 172.21.15.78:0/1965765514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36a0102980 msgr2=0x7f36a0197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:39.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f369ffff640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36a0102980 0x7f36a0197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f369ffff640 1 -- 172.21.15.78:0/1965765514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a019f670 con 0x7f36a0103340 2024-04-25T01:43:39.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f3697fff640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36a0102980 0x7f36a0197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:39.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f369ffff640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a0103340 0x7f36a019f000 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3698005e30 tx=0x7f3698031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:39.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.416+0000 7f369d7fa640 1 -- 172.21.15.78:0/1965765514 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698003c30 con 0x7f36a0103340 2024-04-25T01:43:39.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.417+0000 7f369d7fa640 1 -- 172.21.15.78:0/1965765514 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3698003dd0 con 0x7f36a0103340 2024-04-25T01:43:39.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.417+0000 7f36a5b34640 1 -- 172.21.15.78:0/1965765514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a019f900 con 0x7f36a0103340 2024-04-25T01:43:39.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.417+0000 7f369d7fa640 1 -- 172.21.15.78:0/1965765514 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369802fde0 con 0x7f36a0103340 2024-04-25T01:43:39.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.417+0000 7f36a5b34640 1 -- 172.21.15.78:0/1965765514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a019fe50 con 0x7f36a0103340 2024-04-25T01:43:39.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.417+0000 7f36a5b34640 1 -- 172.21.15.78:0/1965765514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a0108ab0 con 0x7f36a0103340 2024-04-25T01:43:39.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.418+0000 7f369d7fa640 1 -- 172.21.15.78:0/1965765514 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3698004050 con 0x7f36a0103340 2024-04-25T01:43:39.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.419+0000 7f369d7fa640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f368c0a2b90 0x7f368c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:39.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.419+0000 7f369f7fe640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f368c0a2b90 0x7f368c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:39.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.419+0000 7f369d7fa640 1 -- 172.21.15.78:0/1965765514 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f36980f0810 con 0x7f36a0103340 2024-04-25T01:43:39.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.419+0000 7f369f7fe640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f368c0a2b90 0x7f368c0a5050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3690005fd0 tx=0x7f3690005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:39.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.423+0000 7f369d7fa640 1 -- 172.21.15.78:0/1965765514 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f36980bd530 con 0x7f36a0103340 2024-04-25T01:43:39.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.634+0000 7f36a5b34640 1 -- 172.21.15.78:0/1965765514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f36a0062340 con 0x7f36a0103340 2024-04-25T01:43:39.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.976+0000 7f369d7fa640 1 -- 172.21.15.78:0/1965765514 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f36980bce30 con 0x7f36a0103340 2024-04-25T01:43:39.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.978+0000 7f36a5b34640 1 -- 172.21.15.78:0/1965765514 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f368c0a2b90 msgr2=0x7f368c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:39.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.978+0000 7f36a5b34640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f368c0a2b90 0x7f368c0a5050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3690005fd0 tx=0x7f3690005850 comp rx=0 tx=0).stop 2024-04-25T01:43:39.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.978+0000 7f36a5b34640 1 -- 172.21.15.78:0/1965765514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a0103340 msgr2=0x7f36a019f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:39.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.978+0000 7f36a5b34640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a0103340 0x7f36a019f000 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3698005e30 tx=0x7f3698031ee0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.978+0000 7f36a5b34640 1 -- 172.21.15.78:0/1965765514 shutdown_connections 2024-04-25T01:43:39.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.978+0000 7f36a5b34640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f368c0a2b90 0x7f368c0a5050 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.978+0000 7f36a5b34640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36a0103340 0x7f36a019f000 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.978+0000 7f36a5b34640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36a0102980 0x7f36a0197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.978+0000 7f36a5b34640 1 --2- 172.21.15.78:0/1965765514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36a0101790 0x7f36a0197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:39.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.978+0000 7f36a5b34640 1 -- 172.21.15.78:0/1965765514 >> 172.21.15.78:0/1965765514 conn(0x7f36a00fcd20 msgr2=0x7f36a0105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:39.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.978+0000 7f36a5b34640 1 -- 172.21.15.78:0/1965765514 shutdown_connections 2024-04-25T01:43:39.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:39.978+0000 7f36a5b34640 1 -- 172.21.15.78:0/1965765514 wait complete. 2024-04-25T01:43:39.980 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-25T01:43:39.980 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-25T01:43:40.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.089+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/756366436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bf0102250 msgr2=0x7f4bf0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:40.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.089+0000 7f4bf6bd5640 1 --2- 172.21.15.78:0/756366436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bf0102250 0x7f4bf0108ae0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009ca0 tx=0x7f4be802a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.089+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/756366436 shutdown_connections 2024-04-25T01:43:40.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.089+0000 7f4bf6bd5640 1 --2- 172.21.15.78:0/756366436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bf0102250 0x7f4bf0108ae0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.089+0000 7f4bf6bd5640 1 --2- 172.21.15.78:0/756366436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf0101890 0x7f4bf0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.089+0000 7f4bf6bd5640 1 --2- 172.21.15.78:0/756366436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bf01006a0 0x7f4bf0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.089+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/756366436 >> 172.21.15.78:0/756366436 conn(0x7f4bf00fbc10 msgr2=0x7f4bf00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:40.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.089+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/756366436 shutdown_connections 2024-04-25T01:43:40.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.089+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/756366436 wait complete. 2024-04-25T01:43:40.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4bf6bd5640 1 Processor -- start 2024-04-25T01:43:40.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4bf6bd5640 1 -- start start 2024-04-25T01:43:40.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4bf6bd5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf01006a0 0x7f4bf0197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:40.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4bf6bd5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bf0101890 0x7f4bf0197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:40.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4bf6bd5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bf0102250 0x7f4bf019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:40.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4bf6bd5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf010c380 con 0x7f4bf0101890 2024-04-25T01:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4bf6bd5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf010c200 con 0x7f4bf0102250 2024-04-25T01:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4bf6bd5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf010c500 con 0x7f4bf01006a0 2024-04-25T01:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4beffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bf0101890 0x7f4bf0197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4beffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bf0101890 0x7f4bf0197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58396/0 (socket says 172.21.15.78:58396) 2024-04-25T01:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4beffff640 1 -- 172.21.15.78:0/3754702911 learned_addr learned my addr 172.21.15.78:0/3754702911 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4bf4971640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf01006a0 0x7f4bf0197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4beffff640 1 -- 172.21.15.78:0/3754702911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf01006a0 msgr2=0x7f4bf0197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4beffff640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf01006a0 0x7f4bf0197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4beffff640 1 -- 172.21.15.78:0/3754702911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bf0102250 msgr2=0x7f4bf019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4bf5172640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bf0102250 0x7f4bf019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4beffff640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bf0102250 0x7f4bf019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4beffff640 1 -- 172.21.15.78:0/3754702911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf019f650 con 0x7f4bf0101890 2024-04-25T01:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4bf4971640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf01006a0 0x7f4bf0197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:40.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.090+0000 7f4beffff640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bf0101890 0x7f4bf0197f70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00b4a0 tx=0x7f4bdc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:40.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.091+0000 7f4bedffb640 1 -- 172.21.15.78:0/3754702911 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc014070 con 0x7f4bf0101890 2024-04-25T01:43:40.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.091+0000 7f4bedffb640 1 -- 172.21.15.78:0/3754702911 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bdc010af0 con 0x7f4bf0101890 2024-04-25T01:43:40.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.091+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/3754702911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf019f940 con 0x7f4bf0101890 2024-04-25T01:43:40.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.091+0000 7f4bedffb640 1 -- 172.21.15.78:0/3754702911 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc017d10 con 0x7f4bf0101890 2024-04-25T01:43:40.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.091+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/3754702911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf019fec0 con 0x7f4bf0101890 2024-04-25T01:43:40.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.092+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/3754702911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf01a0150 con 0x7f4bf0101890 2024-04-25T01:43:40.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.093+0000 7f4bedffb640 1 -- 172.21.15.78:0/3754702911 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4bdc017070 con 0x7f4bf0101890 2024-04-25T01:43:40.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.093+0000 7f4bedffb640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4be00a2b90 0x7f4be00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:40.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.093+0000 7f4bf4971640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4be00a2b90 0x7f4be00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:40.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.093+0000 7f4bedffb640 1 -- 172.21.15.78:0/3754702911 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4bdc0ce770 con 0x7f4bf0101890 2024-04-25T01:43:40.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.094+0000 7f4bf4971640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4be00a2b90 0x7f4be00a5050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4bf01016f0 tx=0x7f4be400a5e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:40.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.098+0000 7f4bedffb640 1 -- 172.21.15.78:0/3754702911 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4bdc09b490 con 0x7f4bf0101890 2024-04-25T01:43:40.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:39 smithi078 ceph-mon[29469]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-25T01:43:40.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:39 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1876283830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:40.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:39 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/19817090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:40.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:39 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1965765514' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T01:43:40.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:39 smithi152 ceph-mon[32465]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-25T01:43:40.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:39 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1876283830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:40.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:39 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/19817090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:40.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:39 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1965765514' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T01:43:40.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.311+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/3754702911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f4bf01a0320 con 0x7f4bf0101890 2024-04-25T01:43:40.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:39 smithi196 ceph-mon[32978]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-25T01:43:40.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:39 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1876283830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:40.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:39 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/19817090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:40.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:39 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1965765514' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T01:43:40.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.981+0000 7f4bedffb640 1 -- 172.21.15.78:0/3754702911 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f4bdc09ad90 con 0x7f4bf0101890 2024-04-25T01:43:40.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.984+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/3754702911 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4be00a2b90 msgr2=0x7f4be00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:40.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.984+0000 7f4bf6bd5640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4be00a2b90 0x7f4be00a5050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4bf01016f0 tx=0x7f4be400a5e0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.984+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/3754702911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bf0101890 msgr2=0x7f4bf0197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:40.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.984+0000 7f4bf6bd5640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bf0101890 0x7f4bf0197f70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00b4a0 tx=0x7f4bdc00b960 comp rx=0 tx=0).stop 2024-04-25T01:43:40.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.984+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/3754702911 shutdown_connections 2024-04-25T01:43:40.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.984+0000 7f4bf6bd5640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bf0102250 0x7f4bf019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.984+0000 7f4bf6bd5640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4be00a2b90 0x7f4be00a5050 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.984+0000 7f4bf6bd5640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bf0101890 0x7f4bf0197f70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.984+0000 7f4bf6bd5640 1 --2- 172.21.15.78:0/3754702911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf01006a0 0x7f4bf0197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.984+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/3754702911 >> 172.21.15.78:0/3754702911 conn(0x7f4bf00fbc10 msgr2=0x7f4bf0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:40.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.984+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/3754702911 shutdown_connections 2024-04-25T01:43:40.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:40.984+0000 7f4bf6bd5640 1 -- 172.21.15.78:0/3754702911 wait complete. 2024-04-25T01:43:40.991 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T01:43:41.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.106+0000 7fc3a9258640 1 -- 172.21.15.78:0/36385598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3a4102250 msgr2=0x7fc3a4108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.106+0000 7fc3a9258640 1 --2- 172.21.15.78:0/36385598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3a4102250 0x7fc3a4108ae0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009ca0 tx=0x7fc39c02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.106+0000 7fc3a9258640 1 -- 172.21.15.78:0/36385598 shutdown_connections 2024-04-25T01:43:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.106+0000 7fc3a9258640 1 --2- 172.21.15.78:0/36385598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3a4102250 0x7fc3a4108ae0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.106+0000 7fc3a9258640 1 --2- 172.21.15.78:0/36385598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3a4101890 0x7fc3a4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.106+0000 7fc3a9258640 1 --2- 172.21.15.78:0/36385598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a41006a0 0x7fc3a4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.106+0000 7fc3a9258640 1 -- 172.21.15.78:0/36385598 >> 172.21.15.78:0/36385598 conn(0x7fc3a40fbc10 msgr2=0x7fc3a40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.106+0000 7fc3a9258640 1 -- 172.21.15.78:0/36385598 shutdown_connections 2024-04-25T01:43:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.107+0000 7fc3a9258640 1 -- 172.21.15.78:0/36385598 wait complete. 2024-04-25T01:43:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.107+0000 7fc3a9258640 1 Processor -- start 2024-04-25T01:43:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.107+0000 7fc3a9258640 1 -- start start 2024-04-25T01:43:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.107+0000 7fc3a9258640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3a41006a0 0x7fc3a4104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:41.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a9258640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3a4101890 0x7fc3a4102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a9258640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a4102250 0x7fc3a41031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a9258640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a410c410 con 0x7fc3a4101890 2024-04-25T01:43:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a9258640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a410c290 con 0x7fc3a41006a0 2024-04-25T01:43:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a9258640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a410c590 con 0x7fc3a4102250 2024-04-25T01:43:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3a4101890 0x7fc3a4102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3a4101890 0x7fc3a4102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58422/0 (socket says 172.21.15.78:58422) 2024-04-25T01:43:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a259c640 1 -- 172.21.15.78:0/3440431304 learned_addr learned my addr 172.21.15.78:0/3440431304 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a2d9d640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3a41006a0 0x7fc3a4104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a359e640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a4102250 0x7fc3a41031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a259c640 1 -- 172.21.15.78:0/3440431304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a4102250 msgr2=0x7fc3a41031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a259c640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a4102250 0x7fc3a41031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a259c640 1 -- 172.21.15.78:0/3440431304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3a41006a0 msgr2=0x7fc3a4104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a259c640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3a41006a0 0x7fc3a4104630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a259c640 1 -- 172.21.15.78:0/3440431304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a4103a40 con 0x7fc3a4101890 2024-04-25T01:43:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a2d9d640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3a41006a0 0x7fc3a4104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a259c640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3a4101890 0x7fc3a4102c70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc39400b4a0 tx=0x7fc39400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc38bfff640 1 -- 172.21.15.78:0/3440431304 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394014070 con 0x7fc3a4101890 2024-04-25T01:43:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a9258640 1 -- 172.21.15.78:0/3440431304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a41a1ca0 con 0x7fc3a4101890 2024-04-25T01:43:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc3a9258640 1 -- 172.21.15.78:0/3440431304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a41a2210 con 0x7fc3a4101890 2024-04-25T01:43:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc38bfff640 1 -- 172.21.15.78:0/3440431304 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc394010af0 con 0x7fc3a4101890 2024-04-25T01:43:41.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.108+0000 7fc38bfff640 1 -- 172.21.15.78:0/3440431304 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39401e060 con 0x7fc3a4101890 2024-04-25T01:43:41.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.109+0000 7fc3a9258640 1 -- 172.21.15.78:0/3440431304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3900052c0 con 0x7fc3a4101890 2024-04-25T01:43:41.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.111+0000 7fc38bfff640 1 -- 172.21.15.78:0/3440431304 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc394017070 con 0x7fc3a4101890 2024-04-25T01:43:41.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.111+0000 7fc38bfff640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc38c0a2b90 0x7fc38c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:41.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.111+0000 7fc3a2d9d640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc38c0a2b90 0x7fc38c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:41.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.111+0000 7fc38bfff640 1 -- 172.21.15.78:0/3440431304 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc3940ce660 con 0x7fc3a4101890 2024-04-25T01:43:41.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.112+0000 7fc3a2d9d640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc38c0a2b90 0x7fc38c0a5050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3a41016f0 tx=0x7fc398008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:41.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.116+0000 7fc38bfff640 1 -- 172.21.15.78:0/3440431304 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc39409b380 con 0x7fc3a4101890 2024-04-25T01:43:41.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:40 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1965765514' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T01:43:41.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:40 smithi078 ceph-mon[29469]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:41.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:40 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3754702911' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T01:43:41.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:40 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1965765514' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T01:43:41.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:40 smithi152 ceph-mon[32465]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:41.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:40 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3754702911' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T01:43:41.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.328+0000 7fc3a9258640 1 -- 172.21.15.78:0/3440431304 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc3900055b0 con 0x7fc3a4101890 2024-04-25T01:43:41.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:40 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1965765514' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T01:43:41.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:40 smithi196 ceph-mon[32978]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:41.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:40 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3754702911' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T01:43:41.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.989+0000 7fc38bfff640 1 -- 172.21.15.78:0/3440431304 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fc39409ac80 con 0x7fc3a4101890 2024-04-25T01:43:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.992+0000 7fc3a9258640 1 -- 172.21.15.78:0/3440431304 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc38c0a2b90 msgr2=0x7fc38c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.992+0000 7fc3a9258640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc38c0a2b90 0x7fc38c0a5050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3a41016f0 tx=0x7fc398008040 comp rx=0 tx=0).stop 2024-04-25T01:43:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.992+0000 7fc3a9258640 1 -- 172.21.15.78:0/3440431304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3a4101890 msgr2=0x7fc3a4102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.992+0000 7fc3a9258640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3a4101890 0x7fc3a4102c70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc39400b4a0 tx=0x7fc39400b960 comp rx=0 tx=0).stop 2024-04-25T01:43:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.992+0000 7fc3a9258640 1 -- 172.21.15.78:0/3440431304 shutdown_connections 2024-04-25T01:43:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.992+0000 7fc3a9258640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3a4102250 0x7fc3a41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.992+0000 7fc3a9258640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc38c0a2b90 0x7fc38c0a5050 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.992+0000 7fc3a9258640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3a4101890 0x7fc3a4102c70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.992+0000 7fc3a9258640 1 --2- 172.21.15.78:0/3440431304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3a41006a0 0x7fc3a4104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.992+0000 7fc3a9258640 1 -- 172.21.15.78:0/3440431304 >> 172.21.15.78:0/3440431304 conn(0x7fc3a40fbc10 msgr2=0x7fc3a40fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:41.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.992+0000 7fc3a9258640 1 -- 172.21.15.78:0/3440431304 shutdown_connections 2024-04-25T01:43:41.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:41.992+0000 7fc3a9258640 1 -- 172.21.15.78:0/3440431304 wait complete. 2024-04-25T01:43:41.998 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T01:43:42.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.110+0000 7ff9af589640 1 -- 172.21.15.78:0/3063273003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a8100690 msgr2=0x7ff9a8100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.110+0000 7ff9af589640 1 --2- 172.21.15.78:0/3063273003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a8100690 0x7ff9a8100ab0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009900 tx=0x7ff9a402a410 comp rx=0 tx=0).stop 2024-04-25T01:43:42.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.110+0000 7ff9af589640 1 -- 172.21.15.78:0/3063273003 shutdown_connections 2024-04-25T01:43:42.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.110+0000 7ff9af589640 1 --2- 172.21.15.78:0/3063273003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9a8102240 0x7ff9a8108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.110+0000 7ff9af589640 1 --2- 172.21.15.78:0/3063273003 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a8101880 0x7ff9a8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.110+0000 7ff9af589640 1 --2- 172.21.15.78:0/3063273003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a8100690 0x7ff9a8100ab0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.110+0000 7ff9af589640 1 -- 172.21.15.78:0/3063273003 >> 172.21.15.78:0/3063273003 conn(0x7ff9a80fbc00 msgr2=0x7ff9a80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.110+0000 7ff9af589640 1 -- 172.21.15.78:0/3063273003 shutdown_connections 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.110+0000 7ff9af589640 1 -- 172.21.15.78:0/3063273003 wait complete. 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9af589640 1 Processor -- start 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9af589640 1 -- start start 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9af589640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a8100690 0x7ff9a8196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9af589640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9a8101880 0x7ff9a8196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9af589640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a8102240 0x7ff9a819df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9af589640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a810c2e0 con 0x7ff9a8102240 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9af589640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a810c160 con 0x7ff9a8100690 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9af589640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a810c460 con 0x7ff9a8101880 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9adb26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a8102240 0x7ff9a819df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:42.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9adb26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a8102240 0x7ff9a819df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58452/0 (socket says 172.21.15.78:58452) 2024-04-25T01:43:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9adb26640 1 -- 172.21.15.78:0/292800851 learned_addr learned my addr 172.21.15.78:0/292800851 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9ad325640 1 --2- 172.21.15.78:0/292800851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a8100690 0x7ff9a8196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9acb24640 1 --2- 172.21.15.78:0/292800851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9a8101880 0x7ff9a8196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9adb26640 1 -- 172.21.15.78:0/292800851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9a8101880 msgr2=0x7ff9a8196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9adb26640 1 --2- 172.21.15.78:0/292800851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9a8101880 0x7ff9a8196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9adb26640 1 -- 172.21.15.78:0/292800851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a8100690 msgr2=0x7ff9a8196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9adb26640 1 --2- 172.21.15.78:0/292800851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a8100690 0x7ff9a8196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9adb26640 1 -- 172.21.15.78:0/292800851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a819e5a0 con 0x7ff9a8102240 2024-04-25T01:43:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff9adb26640 1 --2- 172.21.15.78:0/292800851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a8102240 0x7ff9a819df30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0007cb0 tx=0x7ff9a000e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.111+0000 7ff99e7fc640 1 -- 172.21.15.78:0/292800851 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000eb30 con 0x7ff9a8102240 2024-04-25T01:43:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.112+0000 7ff99e7fc640 1 -- 172.21.15.78:0/292800851 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9a000ecd0 con 0x7ff9a8102240 2024-04-25T01:43:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.112+0000 7ff9af589640 1 -- 172.21.15.78:0/292800851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a8069dc0 con 0x7ff9a8102240 2024-04-25T01:43:42.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.112+0000 7ff99e7fc640 1 -- 172.21.15.78:0/292800851 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0015650 con 0x7ff9a8102240 2024-04-25T01:43:42.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.112+0000 7ff9af589640 1 -- 172.21.15.78:0/292800851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a806a2c0 con 0x7ff9a8102240 2024-04-25T01:43:42.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.112+0000 7ff9af589640 1 -- 172.21.15.78:0/292800851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a80689b0 con 0x7ff9a8102240 2024-04-25T01:43:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.114+0000 7ff99e7fc640 1 -- 172.21.15.78:0/292800851 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff9a0004a40 con 0x7ff9a8102240 2024-04-25T01:43:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.114+0000 7ff99e7fc640 1 --2- 172.21.15.78:0/292800851 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff9940a2b90 0x7ff9940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.114+0000 7ff9ad325640 1 --2- 172.21.15.78:0/292800851 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff9940a2b90 0x7ff9940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.115+0000 7ff99e7fc640 1 -- 172.21.15.78:0/292800851 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff9a00d10a0 con 0x7ff9a8102240 2024-04-25T01:43:42.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.115+0000 7ff9ad325640 1 --2- 172.21.15.78:0/292800851 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff9940a2b90 0x7ff9940a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4005ab0 tx=0x7ff9a4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:42.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.118+0000 7ff99e7fc640 1 -- 172.21.15.78:0/292800851 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff9a009dd60 con 0x7ff9a8102240 2024-04-25T01:43:42.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:41 smithi078 ceph-mon[29469]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-25T01:43:42.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:41 smithi078 ceph-mon[29469]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T01:43:42.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:41 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3754702911' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T01:43:42.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:41 smithi078 ceph-mon[29469]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T01:43:42.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:41 smithi078 ceph-mon[29469]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:42.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:41 smithi078 ceph-mon[29469]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T01:43:42.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:41 smithi078 ceph-mon[29469]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T01:43:42.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:41 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3440431304' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T01:43:42.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:41 smithi152 ceph-mon[32465]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-25T01:43:42.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:41 smithi152 ceph-mon[32465]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T01:43:42.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:41 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3754702911' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T01:43:42.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:41 smithi152 ceph-mon[32465]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T01:43:42.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:41 smithi152 ceph-mon[32465]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:42.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:41 smithi152 ceph-mon[32465]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T01:43:42.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:41 smithi152 ceph-mon[32465]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T01:43:42.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:41 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3440431304' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T01:43:42.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.332+0000 7ff9af589640 1 -- 172.21.15.78:0/292800851 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7ff9a8107660 con 0x7ff9a8102240 2024-04-25T01:43:42.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:41 smithi196 ceph-mon[32978]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-25T01:43:42.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:41 smithi196 ceph-mon[32978]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T01:43:42.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:41 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3754702911' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T01:43:42.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:41 smithi196 ceph-mon[32978]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T01:43:42.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:41 smithi196 ceph-mon[32978]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T01:43:42.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:41 smithi196 ceph-mon[32978]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T01:43:42.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:41 smithi196 ceph-mon[32978]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T01:43:42.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:41 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3440431304' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T01:43:42.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.409+0000 7ff99e7fc640 1 -- 172.21.15.78:0/292800851 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7ff9a009d660 con 0x7ff9a8102240 2024-04-25T01:43:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.411+0000 7ff9af589640 1 -- 172.21.15.78:0/292800851 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff9940a2b90 msgr2=0x7ff9940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.411+0000 7ff9af589640 1 --2- 172.21.15.78:0/292800851 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff9940a2b90 0x7ff9940a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4005ab0 tx=0x7ff9a4000bc0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.411+0000 7ff9af589640 1 -- 172.21.15.78:0/292800851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a8102240 msgr2=0x7ff9a819df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.411+0000 7ff9af589640 1 --2- 172.21.15.78:0/292800851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a8102240 0x7ff9a819df30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0007cb0 tx=0x7ff9a000e420 comp rx=0 tx=0).stop 2024-04-25T01:43:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.411+0000 7ff9af589640 1 -- 172.21.15.78:0/292800851 shutdown_connections 2024-04-25T01:43:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.411+0000 7ff9af589640 1 --2- 172.21.15.78:0/292800851 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff9940a2b90 0x7ff9940a5050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.411+0000 7ff9af589640 1 --2- 172.21.15.78:0/292800851 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9a8102240 0x7ff9a819df30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.411+0000 7ff9af589640 1 --2- 172.21.15.78:0/292800851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9a8101880 0x7ff9a8196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.411+0000 7ff9af589640 1 --2- 172.21.15.78:0/292800851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a8100690 0x7ff9a8196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.411+0000 7ff9af589640 1 -- 172.21.15.78:0/292800851 >> 172.21.15.78:0/292800851 conn(0x7ff9a80fbc00 msgr2=0x7ff9a8104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.411+0000 7ff9af589640 1 -- 172.21.15.78:0/292800851 shutdown_connections 2024-04-25T01:43:42.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.411+0000 7ff9af589640 1 -- 172.21.15.78:0/292800851 wait complete. 2024-04-25T01:43:42.416 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-25T01:43:42.416 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T01:43:42.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 -- 172.21.15.78:0/985302685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98100680 msgr2=0x7f5d98100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 --2- 172.21.15.78:0/985302685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98100680 0x7f5d98100aa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009900 tx=0x7f5d8802a410 comp rx=0 tx=0).stop 2024-04-25T01:43:42.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 -- 172.21.15.78:0/985302685 shutdown_connections 2024-04-25T01:43:42.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 --2- 172.21.15.78:0/985302685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d98102230 0x7f5d98108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 --2- 172.21.15.78:0/985302685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d98101870 0x7f5d98101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 --2- 172.21.15.78:0/985302685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98100680 0x7f5d98100aa0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 -- 172.21.15.78:0/985302685 >> 172.21.15.78:0/985302685 conn(0x7f5d980fbbf0 msgr2=0x7f5d980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:42.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 -- 172.21.15.78:0/985302685 shutdown_connections 2024-04-25T01:43:42.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 -- 172.21.15.78:0/985302685 wait complete. 2024-04-25T01:43:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 Processor -- start 2024-04-25T01:43:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 -- start start 2024-04-25T01:43:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d98100680 0x7f5d98197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98101870 0x7f5d98198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d98102230 0x7f5d9819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d98069030 con 0x7f5d98101870 2024-04-25T01:43:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5d98068eb0 con 0x7f5d98100680 2024-04-25T01:43:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.527+0000 7f5d9e43f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d980691b0 con 0x7f5d98102230 2024-04-25T01:43:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98101870 0x7f5d98198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98101870 0x7f5d98198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58470/0 (socket says 172.21.15.78:58470) 2024-04-25T01:43:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d977fe640 1 -- 172.21.15.78:0/435945874 learned_addr learned my addr 172.21.15.78:0/435945874 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d97fff640 1 --2- 172.21.15.78:0/435945874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d98100680 0x7f5d98197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:42.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d9c9dc640 1 --2- 172.21.15.78:0/435945874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d98102230 0x7f5d9819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:42.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d977fe640 1 -- 172.21.15.78:0/435945874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d98102230 msgr2=0x7f5d9819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d977fe640 1 --2- 172.21.15.78:0/435945874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d98102230 0x7f5d9819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d977fe640 1 -- 172.21.15.78:0/435945874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d98100680 msgr2=0x7f5d98197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d977fe640 1 --2- 172.21.15.78:0/435945874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d98100680 0x7f5d98197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d977fe640 1 -- 172.21.15.78:0/435945874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9819f710 con 0x7f5d98101870 2024-04-25T01:43:42.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d977fe640 1 --2- 172.21.15.78:0/435945874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98101870 0x7f5d98198030 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00e9f0 tx=0x7f5d8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:42.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d957fa640 1 -- 172.21.15.78:0/435945874 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00cd80 con 0x7f5d98101870 2024-04-25T01:43:42.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d957fa640 1 -- 172.21.15.78:0/435945874 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d8c010910 con 0x7f5d98101870 2024-04-25T01:43:42.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d9e43f640 1 -- 172.21.15.78:0/435945874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9819fa00 con 0x7f5d98101870 2024-04-25T01:43:42.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d957fa640 1 -- 172.21.15.78:0/435945874 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c018990 con 0x7f5d98101870 2024-04-25T01:43:42.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.528+0000 7f5d9e43f640 1 -- 172.21.15.78:0/435945874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9819ff50 con 0x7f5d98101870 2024-04-25T01:43:42.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.529+0000 7f5d9e43f640 1 -- 172.21.15.78:0/435945874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d981078d0 con 0x7f5d98101870 2024-04-25T01:43:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.530+0000 7f5d957fa640 1 -- 172.21.15.78:0/435945874 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5d8c010ac0 con 0x7f5d98101870 2024-04-25T01:43:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.531+0000 7f5d957fa640 1 --2- 172.21.15.78:0/435945874 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d800a2b90 0x7f5d800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:42.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.531+0000 7f5d97fff640 1 --2- 172.21.15.78:0/435945874 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d800a2b90 0x7f5d800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.531+0000 7f5d957fa640 1 -- 172.21.15.78:0/435945874 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5d8c0cf590 con 0x7f5d98101870 2024-04-25T01:43:42.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.531+0000 7f5d97fff640 1 --2- 172.21.15.78:0/435945874 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d800a2b90 0x7f5d800a5050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5d88005ab0 tx=0x7f5d88000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:42.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.535+0000 7f5d957fa640 1 -- 172.21.15.78:0/435945874 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5d8c09bc60 con 0x7f5d98101870 2024-04-25T01:43:42.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:42.738+0000 7f5d9e43f640 1 -- 172.21.15.78:0/435945874 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5d98061b80 con 0x7f5d800a2b90 2024-04-25T01:43:43.006 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:43:42.743+0000 7f41f3cce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:43:43.006 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:43:42.743+0000 7f41f3cce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:43:43.006 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:43:42.743+0000 7f41f3cce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:43:43.006 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:43:42.743+0000 7f41f3cce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:43:43.006 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:43:42.743+0000 7f41f3cce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:43:43.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3440431304' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T01:43:43.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] up:active 2024-04-25T01:43:43.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-mon[29469]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T01:43:43.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-mon[29469]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T01:43:43.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-mon[29469]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T01:43:43.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-mon[29469]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T01:43:43.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/292800851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T01:43:43.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/292800851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T01:43:43.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] up:active 2024-04-25T01:43:43.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-mon[29469]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T01:43:43.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-mon[29469]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:43:43.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-mon[29469]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-25T01:43:43.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:42 smithi078 ceph-mon[29469]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-25T01:43:43.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:42 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3440431304' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T01:43:43.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:42 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] up:active 2024-04-25T01:43:43.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:42 smithi152 ceph-mon[32465]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T01:43:43.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:42 smithi152 ceph-mon[32465]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T01:43:43.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:42 smithi152 ceph-mon[32465]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T01:43:43.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:42 smithi152 ceph-mon[32465]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T01:43:43.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:42 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/292800851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T01:43:43.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:42 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/292800851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T01:43:43.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:42 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] up:active 2024-04-25T01:43:43.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:42 smithi152 ceph-mon[32465]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T01:43:43.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:42 smithi152 ceph-mon[32465]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:43:43.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:42 smithi152 ceph-mon[32465]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-25T01:43:43.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:42 smithi152 ceph-mon[32465]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-25T01:43:43.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:42 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3440431304' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T01:43:43.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:42 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] up:active 2024-04-25T01:43:43.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:42 smithi196 ceph-mon[32978]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T01:43:43.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:42 smithi196 ceph-mon[32978]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T01:43:43.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:42 smithi196 ceph-mon[32978]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T01:43:43.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:42 smithi196 ceph-mon[32978]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T01:43:43.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:42 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/292800851' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T01:43:43.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:42 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/292800851' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T01:43:43.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:42 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] up:active 2024-04-25T01:43:43.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:42 smithi196 ceph-mon[32978]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T01:43:43.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:42 smithi196 ceph-mon[32978]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T01:43:43.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:42 smithi196 ceph-mon[32978]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-25T01:43:43.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:42 smithi196 ceph-mon[32978]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-25T01:43:43.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.449+0000 7f5d957fa640 1 -- 172.21.15.78:0/435945874 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5d98061b80 con 0x7f5d800a2b90 2024-04-25T01:43:43.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.451+0000 7f5d9e43f640 1 -- 172.21.15.78:0/435945874 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d800a2b90 msgr2=0x7f5d800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:43.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.451+0000 7f5d9e43f640 1 --2- 172.21.15.78:0/435945874 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d800a2b90 0x7f5d800a5050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5d88005ab0 tx=0x7f5d88000bc0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.451+0000 7f5d9e43f640 1 -- 172.21.15.78:0/435945874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98101870 msgr2=0x7f5d98198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:43.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.451+0000 7f5d9e43f640 1 --2- 172.21.15.78:0/435945874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98101870 0x7f5d98198030 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00e9f0 tx=0x7f5d8c00eeb0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.451+0000 7f5d9e43f640 1 -- 172.21.15.78:0/435945874 shutdown_connections 2024-04-25T01:43:43.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.451+0000 7f5d9e43f640 1 --2- 172.21.15.78:0/435945874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d98102230 0x7f5d9819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.451+0000 7f5d9e43f640 1 --2- 172.21.15.78:0/435945874 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d800a2b90 0x7f5d800a5050 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.451+0000 7f5d9e43f640 1 --2- 172.21.15.78:0/435945874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d98101870 0x7f5d98198030 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.451+0000 7f5d9e43f640 1 --2- 172.21.15.78:0/435945874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d98100680 0x7f5d98197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.451+0000 7f5d9e43f640 1 -- 172.21.15.78:0/435945874 >> 172.21.15.78:0/435945874 conn(0x7f5d980fbbf0 msgr2=0x7f5d98104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:43.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.451+0000 7f5d9e43f640 1 -- 172.21.15.78:0/435945874 shutdown_connections 2024-04-25T01:43:43.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.451+0000 7f5d9e43f640 1 -- 172.21.15.78:0/435945874 wait complete. 2024-04-25T01:43:43.454 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T01:43:43.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.568+0000 7fe52159e640 1 -- 172.21.15.78:0/3736287732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe51c102150 msgr2=0x7fe51c104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:43.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.568+0000 7fe52159e640 1 --2- 172.21.15.78:0/3736287732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe51c102150 0x7fe51c104540 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009900 tx=0x7fe50c02a410 comp rx=0 tx=0).stop 2024-04-25T01:43:43.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.568+0000 7fe52159e640 1 -- 172.21.15.78:0/3736287732 shutdown_connections 2024-04-25T01:43:43.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.568+0000 7fe52159e640 1 --2- 172.21.15.78:0/3736287732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c1073b0 0x7fe51c1097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.568+0000 7fe52159e640 1 --2- 172.21.15.78:0/3736287732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe51c104a80 0x7fe51c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.568+0000 7fe52159e640 1 --2- 172.21.15.78:0/3736287732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe51c102150 0x7fe51c104540 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.568+0000 7fe52159e640 1 -- 172.21.15.78:0/3736287732 >> 172.21.15.78:0/3736287732 conn(0x7fe51c0fbb00 msgr2=0x7fe51c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:43.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.568+0000 7fe52159e640 1 -- 172.21.15.78:0/3736287732 shutdown_connections 2024-04-25T01:43:43.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.568+0000 7fe52159e640 1 -- 172.21.15.78:0/3736287732 wait complete. 2024-04-25T01:43:43.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe52159e640 1 Processor -- start 2024-04-25T01:43:43.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe52159e640 1 -- start start 2024-04-25T01:43:43.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe52159e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe51c102150 0x7fe51c101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:43.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe52159e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe51c104a80 0x7fe51c100100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:43.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe52159e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c1073b0 0x7fe51c100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:43.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe52159e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c10d3b0 con 0x7fe51c104a80 2024-04-25T01:43:43.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe52159e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c10d230 con 0x7fe51c1073b0 2024-04-25T01:43:43.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe52159e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c10d530 con 0x7fe51c102150 2024-04-25T01:43:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe51b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe51c104a80 0x7fe51c100100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe51b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe51c104a80 0x7fe51c100100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58506/0 (socket says 172.21.15.78:58506) 2024-04-25T01:43:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe51b7fe640 1 -- 172.21.15.78:0/2758215107 learned_addr learned my addr 172.21.15.78:0/2758215107 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe520d9d640 1 --2- 172.21.15.78:0/2758215107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c1073b0 0x7fe51c100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe51bfff640 1 --2- 172.21.15.78:0/2758215107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe51c102150 0x7fe51c101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe51b7fe640 1 -- 172.21.15.78:0/2758215107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe51c102150 msgr2=0x7fe51c101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe51b7fe640 1 --2- 172.21.15.78:0/2758215107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe51c102150 0x7fe51c101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe51b7fe640 1 -- 172.21.15.78:0/2758215107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c1073b0 msgr2=0x7fe51c100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe51b7fe640 1 --2- 172.21.15.78:0/2758215107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c1073b0 0x7fe51c100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.569+0000 7fe51b7fe640 1 -- 172.21.15.78:0/2758215107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c100ed0 con 0x7fe51c104a80 2024-04-25T01:43:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.570+0000 7fe51b7fe640 1 --2- 172.21.15.78:0/2758215107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe51c104a80 0x7fe51c100100 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe51000b450 tx=0x7fe51000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.570+0000 7fe5197fa640 1 -- 172.21.15.78:0/2758215107 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510015020 con 0x7fe51c104a80 2024-04-25T01:43:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.570+0000 7fe5197fa640 1 -- 172.21.15.78:0/2758215107 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe510011d70 con 0x7fe51c104a80 2024-04-25T01:43:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.570+0000 7fe52159e640 1 -- 172.21.15.78:0/2758215107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c1a1b10 con 0x7fe51c104a80 2024-04-25T01:43:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.570+0000 7fe5197fa640 1 -- 172.21.15.78:0/2758215107 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51000f340 con 0x7fe51c104a80 2024-04-25T01:43:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.570+0000 7fe52159e640 1 -- 172.21.15.78:0/2758215107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51c1a2010 con 0x7fe51c104a80 2024-04-25T01:43:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.571+0000 7fe52159e640 1 -- 172.21.15.78:0/2758215107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe51c0fd5f0 con 0x7fe51c104a80 2024-04-25T01:43:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.572+0000 7fe5197fa640 1 -- 172.21.15.78:0/2758215107 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe510018020 con 0x7fe51c104a80 2024-04-25T01:43:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.572+0000 7fe5197fa640 1 --2- 172.21.15.78:0/2758215107 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe5040a2b90 0x7fe5040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.572+0000 7fe51bfff640 1 --2- 172.21.15.78:0/2758215107 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe5040a2b90 0x7fe5040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.572+0000 7fe5197fa640 1 -- 172.21.15.78:0/2758215107 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe5100ced20 con 0x7fe51c104a80 2024-04-25T01:43:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.573+0000 7fe51bfff640 1 --2- 172.21.15.78:0/2758215107 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe5040a2b90 0x7fe5040a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe50c031440 tx=0x7fe50c0315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:43.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.577+0000 7fe5197fa640 1 -- 172.21.15.78:0/2758215107 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe51001f020 con 0x7fe51c104a80 2024-04-25T01:43:43.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.780+0000 7fe52159e640 1 -- 172.21.15.78:0/2758215107 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe51c061b80 con 0x7fe5040a2b90 2024-04-25T01:43:43.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.808+0000 7fe5197fa640 1 -- 172.21.15.78:0/2758215107 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe51c061b80 con 0x7fe5040a2b90 2024-04-25T01:43:43.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.810+0000 7fe52159e640 1 -- 172.21.15.78:0/2758215107 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe5040a2b90 msgr2=0x7fe5040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:43.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.810+0000 7fe52159e640 1 --2- 172.21.15.78:0/2758215107 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe5040a2b90 0x7fe5040a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe50c031440 tx=0x7fe50c0315b0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.810+0000 7fe52159e640 1 -- 172.21.15.78:0/2758215107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe51c104a80 msgr2=0x7fe51c100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.810+0000 7fe52159e640 1 --2- 172.21.15.78:0/2758215107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe51c104a80 0x7fe51c100100 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe51000b450 tx=0x7fe51000b910 comp rx=0 tx=0).stop 2024-04-25T01:43:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.810+0000 7fe52159e640 1 -- 172.21.15.78:0/2758215107 shutdown_connections 2024-04-25T01:43:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.810+0000 7fe52159e640 1 --2- 172.21.15.78:0/2758215107 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c1073b0 0x7fe51c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.810+0000 7fe52159e640 1 --2- 172.21.15.78:0/2758215107 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe5040a2b90 0x7fe5040a5050 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.810+0000 7fe52159e640 1 --2- 172.21.15.78:0/2758215107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe51c104a80 0x7fe51c100100 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.810+0000 7fe52159e640 1 --2- 172.21.15.78:0/2758215107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe51c102150 0x7fe51c101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.810+0000 7fe52159e640 1 -- 172.21.15.78:0/2758215107 >> 172.21.15.78:0/2758215107 conn(0x7fe51c0fbb00 msgr2=0x7fe51c102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.810+0000 7fe52159e640 1 -- 172.21.15.78:0/2758215107 shutdown_connections 2024-04-25T01:43:43.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.810+0000 7fe52159e640 1 -- 172.21.15.78:0/2758215107 wait complete. 2024-04-25T01:43:43.812 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T01:43:43.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.923+0000 7f085a514640 1 -- 172.21.15.78:0/507369512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08540ffe10 msgr2=0x7f0854100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:43.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.923+0000 7f085a514640 1 --2- 172.21.15.78:0/507369512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08540ffe10 0x7f0854100230 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0844009900 tx=0x7f084402a410 comp rx=0 tx=0).stop 2024-04-25T01:43:43.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.923+0000 7f085a514640 1 -- 172.21.15.78:0/507369512 shutdown_connections 2024-04-25T01:43:43.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.923+0000 7f085a514640 1 --2- 172.21.15.78:0/507369512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0854101920 0x7f0854108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.923+0000 7f085a514640 1 --2- 172.21.15.78:0/507369512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854100f60 0x7f08541013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.923+0000 7f085a514640 1 --2- 172.21.15.78:0/507369512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08540ffe10 0x7f0854100230 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.923+0000 7f085a514640 1 -- 172.21.15.78:0/507369512 >> 172.21.15.78:0/507369512 conn(0x7f08540fb380 msgr2=0x7f08540fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.923+0000 7f085a514640 1 -- 172.21.15.78:0/507369512 shutdown_connections 2024-04-25T01:43:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.923+0000 7f085a514640 1 -- 172.21.15.78:0/507369512 wait complete. 2024-04-25T01:43:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.924+0000 7f085a514640 1 Processor -- start 2024-04-25T01:43:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.924+0000 7f085a514640 1 -- start start 2024-04-25T01:43:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.924+0000 7f085a514640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08540ffe10 0x7f08540fd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.924+0000 7f085a514640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854100f60 0x7f085419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.924+0000 7f085a514640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0854101920 0x7f085419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.924+0000 7f085a514640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f085410bb30 con 0x7f0854101920 2024-04-25T01:43:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.924+0000 7f085a514640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f085410b9b0 con 0x7f08540ffe10 2024-04-25T01:43:43.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.924+0000 7f085a514640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f085410bcb0 con 0x7f0854100f60 2024-04-25T01:43:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.924+0000 7f0858ab1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0854101920 0x7f085419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.924+0000 7f0858ab1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0854101920 0x7f085419eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58538/0 (socket says 172.21.15.78:58538) 2024-04-25T01:43:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.924+0000 7f0853fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08540ffe10 0x7f08540fd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.924+0000 7f0858ab1640 1 -- 172.21.15.78:0/3787790914 learned_addr learned my addr 172.21.15.78:0/3787790914 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.925+0000 7f08537fe640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854100f60 0x7f085419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.925+0000 7f0858ab1640 1 -- 172.21.15.78:0/3787790914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854100f60 msgr2=0x7f085419c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.925+0000 7f0858ab1640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854100f60 0x7f085419c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.925+0000 7f0858ab1640 1 -- 172.21.15.78:0/3787790914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08540ffe10 msgr2=0x7f08540fd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.925+0000 7f0858ab1640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08540ffe10 0x7f08540fd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:43.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.925+0000 7f0858ab1640 1 -- 172.21.15.78:0/3787790914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085419f7b0 con 0x7f0854101920 2024-04-25T01:43:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.925+0000 7f0853fff640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08540ffe10 0x7f08540fd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:43:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.925+0000 7f08537fe640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854100f60 0x7f085419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:43:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.925+0000 7f0858ab1640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0854101920 0x7f085419eef0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f084c00eb10 tx=0x7f084c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.925+0000 7f08517fa640 1 -- 172.21.15.78:0/3787790914 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c004b20 con 0x7f0854101920 2024-04-25T01:43:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.925+0000 7f085a514640 1 -- 172.21.15.78:0/3787790914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085419faa0 con 0x7f0854101920 2024-04-25T01:43:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.925+0000 7f08517fa640 1 -- 172.21.15.78:0/3787790914 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c004cc0 con 0x7f0854101920 2024-04-25T01:43:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.925+0000 7f08517fa640 1 -- 172.21.15.78:0/3787790914 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c010760 con 0x7f0854101920 2024-04-25T01:43:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.925+0000 7f085a514640 1 -- 172.21.15.78:0/3787790914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08541a0070 con 0x7f0854101920 2024-04-25T01:43:43.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.926+0000 7f085a514640 1 -- 172.21.15.78:0/3787790914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08541078e0 con 0x7f0854101920 2024-04-25T01:43:43.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.927+0000 7f08517fa640 1 -- 172.21.15.78:0/3787790914 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f084c005020 con 0x7f0854101920 2024-04-25T01:43:43.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.927+0000 7f08517fa640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f083c0a2b90 0x7f083c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:43.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.928+0000 7f0853fff640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f083c0a2b90 0x7f083c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:43.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.928+0000 7f08517fa640 1 -- 172.21.15.78:0/3787790914 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f084c0cf8e0 con 0x7f0854101920 2024-04-25T01:43:43.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.928+0000 7f0853fff640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f083c0a2b90 0x7f083c0a5050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f08440314e0 tx=0x7f0844000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:43.932+0000 7f08517fa640 1 -- 172.21.15.78:0/3787790914 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f084c09c600 con 0x7f0854101920 2024-04-25T01:43:44.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.140+0000 7f085a514640 1 -- 172.21.15.78:0/3787790914 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f0854061b80 con 0x7f083c0a2b90 2024-04-25T01:43:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.145+0000 7f08517fa640 1 -- 172.21.15.78:0/3787790914 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0854061b80 con 0x7f083c0a2b90 2024-04-25T01:43:44.136 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/qa/sv_0/24fa182b-9252-467d-9913-7db2ef154208 2024-04-25T01:43:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.147+0000 7f085a514640 1 -- 172.21.15.78:0/3787790914 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f083c0a2b90 msgr2=0x7f083c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.148+0000 7f085a514640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f083c0a2b90 0x7f083c0a5050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f08440314e0 tx=0x7f0844000bc0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.148+0000 7f085a514640 1 -- 172.21.15.78:0/3787790914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0854101920 msgr2=0x7f085419eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.148+0000 7f085a514640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0854101920 0x7f085419eef0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f084c00eb10 tx=0x7f084c00c2d0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.148+0000 7f085a514640 1 -- 172.21.15.78:0/3787790914 shutdown_connections 2024-04-25T01:43:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.148+0000 7f085a514640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f083c0a2b90 0x7f083c0a5050 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.148+0000 7f085a514640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0854101920 0x7f085419eef0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.148+0000 7f085a514640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0854100f60 0x7f085419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.148+0000 7f085a514640 1 --2- 172.21.15.78:0/3787790914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08540ffe10 0x7f08540fd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.148+0000 7f085a514640 1 -- 172.21.15.78:0/3787790914 >> 172.21.15.78:0/3787790914 conn(0x7f08540fb380 msgr2=0x7f0854104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.148+0000 7f085a514640 1 -- 172.21.15.78:0/3787790914 shutdown_connections 2024-04-25T01:43:44.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.148+0000 7f085a514640 1 -- 172.21.15.78:0/3787790914 wait complete. 2024-04-25T01:43:44.148 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:43 smithi078 ceph-mon[29469]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:44.148 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:43 smithi078 ceph-mon[29469]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T01:43:44.148 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:43 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] up:active 2024-04-25T01:43:44.149 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:43 smithi078 ceph-mon[29469]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T01:43:44.149 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:43 smithi078 ceph-mon[29469]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T01:43:44.149 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:43 smithi078 ceph-mon[29469]: Cluster is now healthy 2024-04-25T01:43:44.149 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:43 smithi078 ceph-mon[29469]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-25T01:43:44.149 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:43 smithi078 ceph-mon[29469]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-25T01:43:44.149 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:43 smithi078 ceph-mon[29469]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-25T01:43:44.150 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T01:43:44.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.261+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/2873075707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b98102250 msgr2=0x7f3b98108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.261+0000 7f3b9e1ee640 1 --2- 172.21.15.78:0/2873075707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b98102250 0x7f3b98108ae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b90009ca0 tx=0x7f3b9002a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.261+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/2873075707 shutdown_connections 2024-04-25T01:43:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.261+0000 7f3b9e1ee640 1 --2- 172.21.15.78:0/2873075707 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b98102250 0x7f3b98108ae0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.261+0000 7f3b9e1ee640 1 --2- 172.21.15.78:0/2873075707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b98101890 0x7f3b98101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.261+0000 7f3b9e1ee640 1 --2- 172.21.15.78:0/2873075707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b981006a0 0x7f3b98100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.261+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/2873075707 >> 172.21.15.78:0/2873075707 conn(0x7f3b980fbc10 msgr2=0x7f3b980fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.262+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/2873075707 shutdown_connections 2024-04-25T01:43:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.262+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/2873075707 wait complete. 2024-04-25T01:43:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.262+0000 7f3b9e1ee640 1 Processor -- start 2024-04-25T01:43:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.262+0000 7f3b9e1ee640 1 -- start start 2024-04-25T01:43:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.262+0000 7f3b9e1ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b981006a0 0x7f3b98197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:44.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.262+0000 7f3b9e1ee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b98101890 0x7f3b98197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.262+0000 7f3b9e1ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b98102250 0x7f3b9819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.262+0000 7f3b9e1ee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9810d440 con 0x7f3b98102250 2024-04-25T01:43:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.262+0000 7f3b9e1ee640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9810d2c0 con 0x7f3b98101890 2024-04-25T01:43:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.262+0000 7f3b9e1ee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9810d5c0 con 0x7f3b981006a0 2024-04-25T01:43:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.262+0000 7f3b97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b98102250 0x7f3b9819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b98102250 0x7f3b9819f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58566/0 (socket says 172.21.15.78:58566) 2024-04-25T01:43:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b97fff640 1 -- 172.21.15.78:0/501685054 learned_addr learned my addr 172.21.15.78:0/501685054 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b977fe640 1 --2- 172.21.15.78:0/501685054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b981006a0 0x7f3b98197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b96ffd640 1 --2- 172.21.15.78:0/501685054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b98101890 0x7f3b98197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b97fff640 1 -- 172.21.15.78:0/501685054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b981006a0 msgr2=0x7f3b98197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b97fff640 1 --2- 172.21.15.78:0/501685054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b981006a0 0x7f3b98197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b97fff640 1 -- 172.21.15.78:0/501685054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b98101890 msgr2=0x7f3b98197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b97fff640 1 --2- 172.21.15.78:0/501685054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b98101890 0x7f3b98197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b97fff640 1 -- 172.21.15.78:0/501685054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9819f690 con 0x7f3b98102250 2024-04-25T01:43:44.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b97fff640 1 --2- 172.21.15.78:0/501685054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b98102250 0x7f3b9819f020 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3b90031ae0 tx=0x7f3b90031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:44.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b94ff9640 1 -- 172.21.15.78:0/501685054 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90003c30 con 0x7f3b98102250 2024-04-25T01:43:44.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b94ff9640 1 -- 172.21.15.78:0/501685054 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b90003dd0 con 0x7f3b98102250 2024-04-25T01:43:44.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/501685054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9819f920 con 0x7f3b98102250 2024-04-25T01:43:44.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b94ff9640 1 -- 172.21.15.78:0/501685054 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9002fcb0 con 0x7f3b98102250 2024-04-25T01:43:44.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.263+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/501685054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b9819fe70 con 0x7f3b98102250 2024-04-25T01:43:44.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.264+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/501685054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b981079c0 con 0x7f3b98102250 2024-04-25T01:43:44.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.265+0000 7f3b94ff9640 1 -- 172.21.15.78:0/501685054 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3b90004050 con 0x7f3b98102250 2024-04-25T01:43:44.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.266+0000 7f3b94ff9640 1 --2- 172.21.15.78:0/501685054 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3b800a2b90 0x7f3b800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:44.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.266+0000 7f3b977fe640 1 --2- 172.21.15.78:0/501685054 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3b800a2b90 0x7f3b800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:44.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.266+0000 7f3b94ff9640 1 -- 172.21.15.78:0/501685054 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3b900f04d0 con 0x7f3b98102250 2024-04-25T01:43:44.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.266+0000 7f3b977fe640 1 --2- 172.21.15.78:0/501685054 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3b800a2b90 0x7f3b800a5050 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3b981016f0 tx=0x7f3b8c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:44.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.270+0000 7f3b94ff9640 1 -- 172.21.15.78:0/501685054 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3b900bd310 con 0x7f3b98102250 2024-04-25T01:43:44.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:43 smithi152 ceph-mon[32465]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:44.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:43 smithi152 ceph-mon[32465]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T01:43:44.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:43 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] up:active 2024-04-25T01:43:44.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:43 smithi152 ceph-mon[32465]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T01:43:44.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:43 smithi152 ceph-mon[32465]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T01:43:44.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:43 smithi152 ceph-mon[32465]: Cluster is now healthy 2024-04-25T01:43:44.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:43 smithi152 ceph-mon[32465]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-25T01:43:44.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:43 smithi152 ceph-mon[32465]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-25T01:43:44.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:43 smithi152 ceph-mon[32465]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-25T01:43:44.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:43 smithi196 ceph-mon[32978]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:44.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:43 smithi196 ceph-mon[32978]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T01:43:44.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:43 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] up:active 2024-04-25T01:43:44.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:43 smithi196 ceph-mon[32978]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T01:43:44.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:43 smithi196 ceph-mon[32978]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T01:43:44.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:43 smithi196 ceph-mon[32978]: Cluster is now healthy 2024-04-25T01:43:44.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:43 smithi196 ceph-mon[32978]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-25T01:43:44.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:43 smithi196 ceph-mon[32978]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-25T01:43:44.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:43 smithi196 ceph-mon[32978]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-25T01:43:44.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.474+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/501685054 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3b98061b80 con 0x7f3b800a2b90 2024-04-25T01:43:44.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.498+0000 7f3b94ff9640 1 -- 172.21.15.78:0/501685054 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3b98061b80 con 0x7f3b800a2b90 2024-04-25T01:43:44.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.500+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/501685054 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3b800a2b90 msgr2=0x7f3b800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.500+0000 7f3b9e1ee640 1 --2- 172.21.15.78:0/501685054 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3b800a2b90 0x7f3b800a5050 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3b981016f0 tx=0x7f3b8c005f90 comp rx=0 tx=0).stop 2024-04-25T01:43:44.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.500+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/501685054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b98102250 msgr2=0x7f3b9819f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.500+0000 7f3b9e1ee640 1 --2- 172.21.15.78:0/501685054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b98102250 0x7f3b9819f020 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3b90031ae0 tx=0x7f3b90031ee0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.500+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/501685054 shutdown_connections 2024-04-25T01:43:44.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.500+0000 7f3b9e1ee640 1 --2- 172.21.15.78:0/501685054 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3b800a2b90 0x7f3b800a5050 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.500+0000 7f3b9e1ee640 1 --2- 172.21.15.78:0/501685054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b98102250 0x7f3b9819f020 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.500+0000 7f3b9e1ee640 1 --2- 172.21.15.78:0/501685054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b98101890 0x7f3b98197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.500+0000 7f3b9e1ee640 1 --2- 172.21.15.78:0/501685054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b981006a0 0x7f3b98197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.500+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/501685054 >> 172.21.15.78:0/501685054 conn(0x7f3b980fbc10 msgr2=0x7f3b98104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:44.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.500+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/501685054 shutdown_connections 2024-04-25T01:43:44.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.500+0000 7f3b9e1ee640 1 -- 172.21.15.78:0/501685054 wait complete. 2024-04-25T01:43:44.502 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T01:43:44.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.613+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2017718318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e40101870 msgr2=0x7f9e40101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.613+0000 7f9e45fb7640 1 --2- 172.21.15.78:0/2017718318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e40101870 0x7f9e40101cf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009900 tx=0x7f9e2802a410 comp rx=0 tx=0).stop 2024-04-25T01:43:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.613+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2017718318 shutdown_connections 2024-04-25T01:43:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.613+0000 7f9e45fb7640 1 --2- 172.21.15.78:0/2017718318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e40102230 0x7f9e40108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.613+0000 7f9e45fb7640 1 --2- 172.21.15.78:0/2017718318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e40101870 0x7f9e40101cf0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.613+0000 7f9e45fb7640 1 --2- 172.21.15.78:0/2017718318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e40100680 0x7f9e40100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.613+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2017718318 >> 172.21.15.78:0/2017718318 conn(0x7f9e400fbbf0 msgr2=0x7f9e400fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.613+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2017718318 shutdown_connections 2024-04-25T01:43:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.613+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2017718318 wait complete. 2024-04-25T01:43:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e45fb7640 1 Processor -- start 2024-04-25T01:43:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e45fb7640 1 -- start start 2024-04-25T01:43:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e45fb7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e40100680 0x7f9e40104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e45fb7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e40101870 0x7f9e40102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:44.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e45fb7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e40102230 0x7f9e40103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e45fb7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4010c3b0 con 0x7f9e40102230 2024-04-25T01:43:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e45fb7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4010c230 con 0x7f9e40100680 2024-04-25T01:43:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e45fb7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4010c530 con 0x7f9e40101870 2024-04-25T01:43:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e3ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e40102230 0x7f9e40103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e3f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e40100680 0x7f9e40104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e3ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e40102230 0x7f9e40103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58588/0 (socket says 172.21.15.78:58588) 2024-04-25T01:43:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e3ffff640 1 -- 172.21.15.78:0/2561381067 learned_addr learned my addr 172.21.15.78:0/2561381067 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e36ffd640 1 --2- 172.21.15.78:0/2561381067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e40101870 0x7f9e40102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e3ffff640 1 -- 172.21.15.78:0/2561381067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e40101870 msgr2=0x7f9e40102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e3ffff640 1 --2- 172.21.15.78:0/2561381067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e40101870 0x7f9e40102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e3ffff640 1 -- 172.21.15.78:0/2561381067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e40100680 msgr2=0x7f9e40104610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e3ffff640 1 --2- 172.21.15.78:0/2561381067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e40100680 0x7f9e40104610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.614+0000 7f9e3ffff640 1 -- 172.21.15.78:0/2561381067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e40103a20 con 0x7f9e40102230 2024-04-25T01:43:44.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.615+0000 7f9e3f7fe640 1 --2- 172.21.15.78:0/2561381067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e40100680 0x7f9e40104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:43:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.615+0000 7f9e3ffff640 1 --2- 172.21.15.78:0/2561381067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e40102230 0x7f9e40103190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800b970 tx=0x7f9e3800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.615+0000 7f9e3d7fa640 1 -- 172.21.15.78:0/2561381067 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38019070 con 0x7f9e40102230 2024-04-25T01:43:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.615+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2561381067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e401a1ca0 con 0x7f9e40102230 2024-04-25T01:43:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.615+0000 7f9e3d7fa640 1 -- 172.21.15.78:0/2561381067 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e3800cb40 con 0x7f9e40102230 2024-04-25T01:43:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.615+0000 7f9e3d7fa640 1 -- 172.21.15.78:0/2561381067 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38007650 con 0x7f9e40102230 2024-04-25T01:43:44.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.615+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2561381067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e401a21c0 con 0x7f9e40102230 2024-04-25T01:43:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.616+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2561381067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e40068560 con 0x7f9e40102230 2024-04-25T01:43:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.617+0000 7f9e3d7fa640 1 -- 172.21.15.78:0/2561381067 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9e3801c070 con 0x7f9e40102230 2024-04-25T01:43:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.618+0000 7f9e3d7fa640 1 --2- 172.21.15.78:0/2561381067 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f9e2c0a2b90 0x7f9e2c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.618+0000 7f9e3f7fe640 1 --2- 172.21.15.78:0/2561381067 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f9e2c0a2b90 0x7f9e2c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.618+0000 7f9e3d7fa640 1 -- 172.21.15.78:0/2561381067 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9e380ce880 con 0x7f9e40102230 2024-04-25T01:43:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.618+0000 7f9e3f7fe640 1 --2- 172.21.15.78:0/2561381067 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f9e2c0a2b90 0x7f9e2c0a5050 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9e401016d0 tx=0x7f9e30005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:44.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.622+0000 7f9e3d7fa640 1 -- 172.21.15.78:0/2561381067 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9e3809af50 con 0x7f9e40102230 2024-04-25T01:43:44.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.827+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2561381067 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f9e400fddc0 con 0x7f9e2c0a2b90 2024-04-25T01:43:44.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.831+0000 7f9e3d7fa640 1 -- 172.21.15.78:0/2561381067 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9e400fddc0 con 0x7f9e2c0a2b90 2024-04-25T01:43:44.822 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/qa/sv_1/99d6b083-a499-4291-8c2d-20d599de76b3 2024-04-25T01:43:44.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.833+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2561381067 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f9e2c0a2b90 msgr2=0x7f9e2c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.833+0000 7f9e45fb7640 1 --2- 172.21.15.78:0/2561381067 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f9e2c0a2b90 0x7f9e2c0a5050 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9e401016d0 tx=0x7f9e30005f90 comp rx=0 tx=0).stop 2024-04-25T01:43:44.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.833+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2561381067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e40102230 msgr2=0x7f9e40103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.833+0000 7f9e45fb7640 1 --2- 172.21.15.78:0/2561381067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e40102230 0x7f9e40103190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800b970 tx=0x7f9e3800be30 comp rx=0 tx=0).stop 2024-04-25T01:43:44.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.833+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2561381067 shutdown_connections 2024-04-25T01:43:44.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.833+0000 7f9e45fb7640 1 --2- 172.21.15.78:0/2561381067 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f9e2c0a2b90 0x7f9e2c0a5050 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.833+0000 7f9e45fb7640 1 --2- 172.21.15.78:0/2561381067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e40102230 0x7f9e40103190 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.833+0000 7f9e45fb7640 1 --2- 172.21.15.78:0/2561381067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e40101870 0x7f9e40102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.833+0000 7f9e45fb7640 1 --2- 172.21.15.78:0/2561381067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e40100680 0x7f9e40104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.833+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2561381067 >> 172.21.15.78:0/2561381067 conn(0x7f9e400fbbf0 msgr2=0x7f9e400fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:44.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.833+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2561381067 shutdown_connections 2024-04-25T01:43:44.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.833+0000 7f9e45fb7640 1 -- 172.21.15.78:0/2561381067 wait complete. 2024-04-25T01:43:44.836 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:43:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.942+0000 7f3a4f238640 1 -- 172.21.15.78:0/3190312550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a48100680 msgr2=0x7f3a48100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.942+0000 7f3a4f238640 1 --2- 172.21.15.78:0/3190312550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a48100680 0x7f3a48100aa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009900 tx=0x7f3a3c02a410 comp rx=0 tx=0).stop 2024-04-25T01:43:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.942+0000 7f3a4f238640 1 -- 172.21.15.78:0/3190312550 shutdown_connections 2024-04-25T01:43:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.942+0000 7f3a4f238640 1 --2- 172.21.15.78:0/3190312550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a48102230 0x7f3a48108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.942+0000 7f3a4f238640 1 --2- 172.21.15.78:0/3190312550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a48101870 0x7f3a48101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.942+0000 7f3a4f238640 1 --2- 172.21.15.78:0/3190312550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a48100680 0x7f3a48100aa0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.942+0000 7f3a4f238640 1 -- 172.21.15.78:0/3190312550 >> 172.21.15.78:0/3190312550 conn(0x7f3a480fbbf0 msgr2=0x7f3a480fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.942+0000 7f3a4f238640 1 -- 172.21.15.78:0/3190312550 shutdown_connections 2024-04-25T01:43:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.942+0000 7f3a4f238640 1 -- 172.21.15.78:0/3190312550 wait complete. 2024-04-25T01:43:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4f238640 1 Processor -- start 2024-04-25T01:43:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4f238640 1 -- start start 2024-04-25T01:43:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4f238640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a48100680 0x7f3a48197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4f238640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a48101870 0x7f3a48198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4f238640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a48102230 0x7f3a4819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4f238640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4810d420 con 0x7f3a48102230 2024-04-25T01:43:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4f238640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4810d2a0 con 0x7f3a48100680 2024-04-25T01:43:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4f238640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4810d5a0 con 0x7f3a48101870 2024-04-25T01:43:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4d7d5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a48102230 0x7f3a4819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4d7d5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a48102230 0x7f3a4819f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58600/0 (socket says 172.21.15.78:58600) 2024-04-25T01:43:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4cfd4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a48100680 0x7f3a48197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4d7d5640 1 -- 172.21.15.78:0/33290992 learned_addr learned my addr 172.21.15.78:0/33290992 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a47fff640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a48101870 0x7f3a48198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4d7d5640 1 -- 172.21.15.78:0/33290992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a48101870 msgr2=0x7f3a48198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4d7d5640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a48101870 0x7f3a48198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4d7d5640 1 -- 172.21.15.78:0/33290992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a48100680 msgr2=0x7f3a48197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4d7d5640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a48100680 0x7f3a48197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4d7d5640 1 -- 172.21.15.78:0/33290992 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4819f720 con 0x7f3a48102230 2024-04-25T01:43:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a47fff640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a48101870 0x7f3a48198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:44.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.943+0000 7f3a4cfd4640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a48100680 0x7f3a48197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:43:44.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.944+0000 7f3a4d7d5640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a48102230 0x7f3a4819f0b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000efd0 tx=0x7f3a4000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:44.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.944+0000 7f3a45ffb640 1 -- 172.21.15.78:0/33290992 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40003b70 con 0x7f3a48102230 2024-04-25T01:43:44.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.944+0000 7f3a4f238640 1 -- 172.21.15.78:0/33290992 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4819fa10 con 0x7f3a48102230 2024-04-25T01:43:44.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.944+0000 7f3a45ffb640 1 -- 172.21.15.78:0/33290992 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a40003d10 con 0x7f3a48102230 2024-04-25T01:43:44.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.944+0000 7f3a45ffb640 1 -- 172.21.15.78:0/33290992 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a400109f0 con 0x7f3a48102230 2024-04-25T01:43:44.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.944+0000 7f3a4f238640 1 -- 172.21.15.78:0/33290992 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4819ff10 con 0x7f3a48102230 2024-04-25T01:43:44.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.945+0000 7f3a4f238640 1 -- 172.21.15.78:0/33290992 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a48107a20 con 0x7f3a48102230 2024-04-25T01:43:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.946+0000 7f3a45ffb640 1 -- 172.21.15.78:0/33290992 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3a40004020 con 0x7f3a48102230 2024-04-25T01:43:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.946+0000 7f3a45ffb640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3a380a2b90 0x7f3a380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.946+0000 7f3a4cfd4640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3a380a2b90 0x7f3a380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.946+0000 7f3a45ffb640 1 -- 172.21.15.78:0/33290992 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3a400cf770 con 0x7f3a48102230 2024-04-25T01:43:44.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.947+0000 7f3a4cfd4640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3a380a2b90 0x7f3a380a5050 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c005ab0 tx=0x7f3a3c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:44.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:44.951+0000 7f3a45ffb640 1 -- 172.21.15.78:0/33290992 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3a4009c490 con 0x7f3a48102230 2024-04-25T01:43:45.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.166+0000 7f3a4f238640 1 -- 172.21.15.78:0/33290992 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a4802d170 con 0x7f3a48102230 2024-04-25T01:43:45.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.167+0000 7f3a45ffb640 1 -- 172.21.15.78:0/33290992 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f3a4009bd90 con 0x7f3a48102230 2024-04-25T01:43:45.159 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:45.159 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:43:17.355192+0000","modified":"2024-04-25T01:43:44.435289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24455,"mds_2":24448,"mds_3":24392,"mds_4":24379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1337519666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1337519666},{"type":"v1","addr":"172.21.15.78:6839","nonce":1337519666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1963410683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1963410683},{"type":"v1","addr":"172.21.15.78:6841","nonce":1963410683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24379":{"gid":24379,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1969344843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1969344843},{"type":"v1","addr":"172.21.15.152:6839","nonce":1969344843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24379,24491,24455,24392,24448]},"id":1}]} 2024-04-25T01:43:45.161 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-04-25T01:43:45.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.169+0000 7f3a4f238640 1 -- 172.21.15.78:0/33290992 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3a380a2b90 msgr2=0x7f3a380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.169+0000 7f3a4f238640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3a380a2b90 0x7f3a380a5050 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c005ab0 tx=0x7f3a3c000bc0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.169+0000 7f3a4f238640 1 -- 172.21.15.78:0/33290992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a48102230 msgr2=0x7f3a4819f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.169+0000 7f3a4f238640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a48102230 0x7f3a4819f0b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000efd0 tx=0x7f3a4000c6a0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.169+0000 7f3a4f238640 1 -- 172.21.15.78:0/33290992 shutdown_connections 2024-04-25T01:43:45.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.169+0000 7f3a4f238640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3a380a2b90 0x7f3a380a5050 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.169+0000 7f3a4f238640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a48102230 0x7f3a4819f0b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.169+0000 7f3a4f238640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a48101870 0x7f3a48198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.169+0000 7f3a4f238640 1 --2- 172.21.15.78:0/33290992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a48100680 0x7f3a48197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.169+0000 7f3a4f238640 1 -- 172.21.15.78:0/33290992 >> 172.21.15.78:0/33290992 conn(0x7f3a480fbbf0 msgr2=0x7f3a48104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:45.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.169+0000 7f3a4f238640 1 -- 172.21.15.78:0/33290992 shutdown_connections 2024-04-25T01:43:45.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.169+0000 7f3a4f238640 1 -- 172.21.15.78:0/33290992 wait complete. 2024-04-25T01:43:45.170 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:44 smithi078 ceph-mon[29469]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:45.170 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:44 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] up:active 2024-04-25T01:43:45.170 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:44 smithi078 ceph-mon[29469]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-25T01:43:45.170 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:44 smithi078 ceph-mon[29469]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T01:43:45.172 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:43:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.279+0000 7fe711314640 1 -- 172.21.15.78:0/411364941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe70c103340 msgr2=0x7fe70c109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.279+0000 7fe711314640 1 --2- 172.21.15.78:0/411364941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe70c103340 0x7fe70c109bd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe704009ca0 tx=0x7fe70402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.279+0000 7fe711314640 1 -- 172.21.15.78:0/411364941 shutdown_connections 2024-04-25T01:43:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.279+0000 7fe711314640 1 --2- 172.21.15.78:0/411364941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe70c103340 0x7fe70c109bd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.279+0000 7fe711314640 1 --2- 172.21.15.78:0/411364941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe70c102980 0x7fe70c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.279+0000 7fe711314640 1 --2- 172.21.15.78:0/411364941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe70c101790 0x7fe70c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.280+0000 7fe711314640 1 -- 172.21.15.78:0/411364941 >> 172.21.15.78:0/411364941 conn(0x7fe70c0fcd20 msgr2=0x7fe70c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.280+0000 7fe711314640 1 -- 172.21.15.78:0/411364941 shutdown_connections 2024-04-25T01:43:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.280+0000 7fe711314640 1 -- 172.21.15.78:0/411364941 wait complete. 2024-04-25T01:43:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.280+0000 7fe711314640 1 Processor -- start 2024-04-25T01:43:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.280+0000 7fe711314640 1 -- start start 2024-04-25T01:43:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.280+0000 7fe711314640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe70c101790 0x7fe70c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.280+0000 7fe711314640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe70c102980 0x7fe70c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.280+0000 7fe711314640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe70c103340 0x7fe70c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.280+0000 7fe711314640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c10d410 con 0x7fe70c101790 2024-04-25T01:43:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.280+0000 7fe711314640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c10d290 con 0x7fe70c103340 2024-04-25T01:43:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.280+0000 7fe711314640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c10d590 con 0x7fe70c102980 2024-04-25T01:43:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.280+0000 7fe70affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe70c101790 0x7fe70c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe70affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe70c101790 0x7fe70c197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58612/0 (socket says 172.21.15.78:58612) 2024-04-25T01:43:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe70affd640 1 -- 172.21.15.78:0/489610415 learned_addr learned my addr 172.21.15.78:0/489610415 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe70b7fe640 1 --2- 172.21.15.78:0/489610415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe70c103340 0x7fe70c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe70a7fc640 1 --2- 172.21.15.78:0/489610415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe70c102980 0x7fe70c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe70affd640 1 -- 172.21.15.78:0/489610415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe70c102980 msgr2=0x7fe70c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe70affd640 1 --2- 172.21.15.78:0/489610415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe70c102980 0x7fe70c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe70affd640 1 -- 172.21.15.78:0/489610415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe70c103340 msgr2=0x7fe70c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe70affd640 1 --2- 172.21.15.78:0/489610415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe70c103340 0x7fe70c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe70affd640 1 -- 172.21.15.78:0/489610415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c19f640 con 0x7fe70c101790 2024-04-25T01:43:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe70affd640 1 --2- 172.21.15.78:0/489610415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe70c101790 0x7fe70c197a20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00c970 tx=0x7fe6fc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe6f3fff640 1 -- 172.21.15.78:0/489610415 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc004220 con 0x7fe70c101790 2024-04-25T01:43:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe6f3fff640 1 -- 172.21.15.78:0/489610415 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6fc0043c0 con 0x7fe70c101790 2024-04-25T01:43:45.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe711314640 1 -- 172.21.15.78:0/489610415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70c19f930 con 0x7fe70c101790 2024-04-25T01:43:45.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe6f3fff640 1 -- 172.21.15.78:0/489610415 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc004a00 con 0x7fe70c101790 2024-04-25T01:43:45.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.281+0000 7fe711314640 1 -- 172.21.15.78:0/489610415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70c19fe30 con 0x7fe70c101790 2024-04-25T01:43:45.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.282+0000 7fe711314640 1 -- 172.21.15.78:0/489610415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70c0689b0 con 0x7fe70c101790 2024-04-25T01:43:45.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.283+0000 7fe6f3fff640 1 -- 172.21.15.78:0/489610415 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe6fc0079b0 con 0x7fe70c101790 2024-04-25T01:43:45.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.284+0000 7fe6f3fff640 1 --2- 172.21.15.78:0/489610415 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe6f40a2b90 0x7fe6f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:45.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.284+0000 7fe70a7fc640 1 --2- 172.21.15.78:0/489610415 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe6f40a2b90 0x7fe6f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:45.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.284+0000 7fe6f3fff640 1 -- 172.21.15.78:0/489610415 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe6fc0cf020 con 0x7fe70c101790 2024-04-25T01:43:45.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.284+0000 7fe70a7fc640 1 --2- 172.21.15.78:0/489610415 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe6f40a2b90 0x7fe6f40a5050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe70c198f30 tx=0x7fe700005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:45.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.288+0000 7fe6f3fff640 1 -- 172.21.15.78:0/489610415 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe6fc014030 con 0x7fe70c101790 2024-04-25T01:43:45.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:44 smithi152 ceph-mon[32465]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:45.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:44 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] up:active 2024-04-25T01:43:45.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:44 smithi152 ceph-mon[32465]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-25T01:43:45.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:44 smithi152 ceph-mon[32465]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T01:43:45.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:44 smithi196 ceph-mon[32978]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:45.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:44 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] up:active 2024-04-25T01:43:45.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:44 smithi196 ceph-mon[32978]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-25T01:43:45.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:44 smithi196 ceph-mon[32978]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T01:43:45.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.461+0000 7fe6f3fff640 1 -- 172.21.15.78:0/489610415 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe6fc097cb0 con 0x7fe70c101790 2024-04-25T01:43:45.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.475+0000 7fe711314640 1 -- 172.21.15.78:0/489610415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe70c062370 con 0x7fe70c101790 2024-04-25T01:43:45.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.477+0000 7fe6f3fff640 1 -- 172.21.15.78:0/489610415 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24151 (secure 0 0 0) 0x7fe6fc097020 con 0x7fe70c101790 2024-04-25T01:43:45.468 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:45.468 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:43:21.781660+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:43:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.479+0000 7fe711314640 1 -- 172.21.15.78:0/489610415 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe6f40a2b90 msgr2=0x7fe6f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.479+0000 7fe711314640 1 --2- 172.21.15.78:0/489610415 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe6f40a2b90 0x7fe6f40a5050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe70c198f30 tx=0x7fe700005f90 comp rx=0 tx=0).stop 2024-04-25T01:43:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.479+0000 7fe711314640 1 -- 172.21.15.78:0/489610415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe70c101790 msgr2=0x7fe70c197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.479+0000 7fe711314640 1 --2- 172.21.15.78:0/489610415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe70c101790 0x7fe70c197a20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00c970 tx=0x7fe6fc00ce30 comp rx=0 tx=0).stop 2024-04-25T01:43:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.479+0000 7fe711314640 1 -- 172.21.15.78:0/489610415 shutdown_connections 2024-04-25T01:43:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.479+0000 7fe711314640 1 --2- 172.21.15.78:0/489610415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe70c103340 0x7fe70c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.479+0000 7fe711314640 1 --2- 172.21.15.78:0/489610415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe70c102980 0x7fe70c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.479+0000 7fe711314640 1 --2- 172.21.15.78:0/489610415 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe6f40a2b90 0x7fe6f40a5050 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.479+0000 7fe711314640 1 --2- 172.21.15.78:0/489610415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe70c101790 0x7fe70c197a20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.479+0000 7fe711314640 1 -- 172.21.15.78:0/489610415 >> 172.21.15.78:0/489610415 conn(0x7fe70c0fcd20 msgr2=0x7fe70c105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:45.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.479+0000 7fe711314640 1 -- 172.21.15.78:0/489610415 shutdown_connections 2024-04-25T01:43:45.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.479+0000 7fe711314640 1 -- 172.21.15.78:0/489610415 wait complete. 2024-04-25T01:43:45.481 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T01:43:45.482 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:43:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.588+0000 7f0710cfc640 1 -- 172.21.15.78:0/3080096162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f070c1022a0 msgr2=0x7f070c104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.588+0000 7f0710cfc640 1 --2- 172.21.15.78:0/3080096162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f070c1022a0 0x7f070c104690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009ca0 tx=0x7f06fc02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.588+0000 7f0710cfc640 1 -- 172.21.15.78:0/3080096162 shutdown_connections 2024-04-25T01:43:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.588+0000 7f0710cfc640 1 --2- 172.21.15.78:0/3080096162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070c107500 0x7f070c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.588+0000 7f0710cfc640 1 --2- 172.21.15.78:0/3080096162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f070c104bd0 0x7f070c106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.588+0000 7f0710cfc640 1 --2- 172.21.15.78:0/3080096162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f070c1022a0 0x7f070c104690 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.588+0000 7f0710cfc640 1 -- 172.21.15.78:0/3080096162 >> 172.21.15.78:0/3080096162 conn(0x7f070c0fbbf0 msgr2=0x7f070c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.588+0000 7f0710cfc640 1 -- 172.21.15.78:0/3080096162 shutdown_connections 2024-04-25T01:43:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.588+0000 7f0710cfc640 1 -- 172.21.15.78:0/3080096162 wait complete. 2024-04-25T01:43:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.589+0000 7f0710cfc640 1 Processor -- start 2024-04-25T01:43:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.589+0000 7f0710cfc640 1 -- start start 2024-04-25T01:43:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.589+0000 7f0710cfc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f070c1022a0 0x7f070c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.589+0000 7f0710cfc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f070c104bd0 0x7f070c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.589+0000 7f0710cfc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070c107500 0x7f070c19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.589+0000 7f0710cfc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f070c10d410 con 0x7f070c104bd0 2024-04-25T01:43:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.589+0000 7f0710cfc640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f070c10d290 con 0x7f070c1022a0 2024-04-25T01:43:45.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.589+0000 7f0710cfc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f070c10d590 con 0x7f070c107500 2024-04-25T01:43:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.589+0000 7f0709d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f070c104bd0 0x7f070c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.589+0000 7f0709d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f070c104bd0 0x7f070c197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58630/0 (socket says 172.21.15.78:58630) 2024-04-25T01:43:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.589+0000 7f0709d9b640 1 -- 172.21.15.78:0/1219849557 learned_addr learned my addr 172.21.15.78:0/1219849557 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.590+0000 7f070a59c640 1 --2- 172.21.15.78:0/1219849557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f070c1022a0 0x7f070c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.590+0000 7f070ad9d640 1 --2- 172.21.15.78:0/1219849557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070c107500 0x7f070c19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.590+0000 7f0709d9b640 1 -- 172.21.15.78:0/1219849557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070c107500 msgr2=0x7f070c19f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.590+0000 7f0709d9b640 1 --2- 172.21.15.78:0/1219849557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070c107500 0x7f070c19f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.590+0000 7f0709d9b640 1 -- 172.21.15.78:0/1219849557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f070c1022a0 msgr2=0x7f070c197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.590+0000 7f0709d9b640 1 --2- 172.21.15.78:0/1219849557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f070c1022a0 0x7f070c197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.590+0000 7f0709d9b640 1 -- 172.21.15.78:0/1219849557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c19f6a0 con 0x7f070c104bd0 2024-04-25T01:43:45.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.590+0000 7f0709d9b640 1 --2- 172.21.15.78:0/1219849557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f070c104bd0 0x7f070c197fc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0700005fd0 tx=0x7f070000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:45.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.590+0000 7f06fb7fe640 1 -- 172.21.15.78:0/1219849557 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700017070 con 0x7f070c104bd0 2024-04-25T01:43:45.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.590+0000 7f06fb7fe640 1 -- 172.21.15.78:0/1219849557 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0700009c30 con 0x7f070c104bd0 2024-04-25T01:43:45.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.590+0000 7f0710cfc640 1 -- 172.21.15.78:0/1219849557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070c19f990 con 0x7f070c104bd0 2024-04-25T01:43:45.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.590+0000 7f06fb7fe640 1 -- 172.21.15.78:0/1219849557 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700010460 con 0x7f070c104bd0 2024-04-25T01:43:45.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.590+0000 7f0710cfc640 1 -- 172.21.15.78:0/1219849557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070c19fee0 con 0x7f070c104bd0 2024-04-25T01:43:45.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.591+0000 7f0710cfc640 1 -- 172.21.15.78:0/1219849557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f070c0fd6e0 con 0x7f070c104bd0 2024-04-25T01:43:45.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.592+0000 7f06fb7fe640 1 -- 172.21.15.78:0/1219849557 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0700003e70 con 0x7f070c104bd0 2024-04-25T01:43:45.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.593+0000 7f06fb7fe640 1 --2- 172.21.15.78:0/1219849557 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f06f40a2ca0 0x7f06f40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:45.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.593+0000 7f070a59c640 1 --2- 172.21.15.78:0/1219849557 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f06f40a2ca0 0x7f06f40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:45.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.593+0000 7f06fb7fe640 1 -- 172.21.15.78:0/1219849557 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f07000cf910 con 0x7f070c104bd0 2024-04-25T01:43:45.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.593+0000 7f070a59c640 1 --2- 172.21.15.78:0/1219849557 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f06f40a2ca0 0x7f06f40a5160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f06fc006010 tx=0x7f06fc000f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:45.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.597+0000 7f06fb7fe640 1 -- 172.21.15.78:0/1219849557 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0700020030 con 0x7f070c104bd0 2024-04-25T01:43:45.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.769+0000 7f0710cfc640 1 -- 172.21.15.78:0/1219849557 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f070c061b80 con 0x7f06f40a2ca0 2024-04-25T01:43:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.779+0000 7f06fb7fe640 1 -- 172.21.15.78:0/1219849557 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340583 (secure 0 0 0) 0x7f070c061b80 con 0x7f06f40a2ca0 2024-04-25T01:43:45.775 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-04-25T01:43:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.782+0000 7f0710cfc640 1 -- 172.21.15.78:0/1219849557 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f06f40a2ca0 msgr2=0x7f06f40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.782+0000 7f0710cfc640 1 --2- 172.21.15.78:0/1219849557 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f06f40a2ca0 0x7f06f40a5160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f06fc006010 tx=0x7f06fc000f40 comp rx=0 tx=0).stop 2024-04-25T01:43:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.782+0000 7f0710cfc640 1 -- 172.21.15.78:0/1219849557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f070c104bd0 msgr2=0x7f070c197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.782+0000 7f0710cfc640 1 --2- 172.21.15.78:0/1219849557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f070c104bd0 0x7f070c197fc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0700005fd0 tx=0x7f070000dde0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.783+0000 7f0710cfc640 1 -- 172.21.15.78:0/1219849557 shutdown_connections 2024-04-25T01:43:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.783+0000 7f0710cfc640 1 --2- 172.21.15.78:0/1219849557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070c107500 0x7f070c19f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.783+0000 7f0710cfc640 1 --2- 172.21.15.78:0/1219849557 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f06f40a2ca0 0x7f06f40a5160 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.783+0000 7f0710cfc640 1 --2- 172.21.15.78:0/1219849557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f070c104bd0 0x7f070c197fc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.783+0000 7f0710cfc640 1 --2- 172.21.15.78:0/1219849557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f070c1022a0 0x7f070c197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.783+0000 7f0710cfc640 1 -- 172.21.15.78:0/1219849557 >> 172.21.15.78:0/1219849557 conn(0x7f070c0fbbf0 msgr2=0x7f070c102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.783+0000 7f0710cfc640 1 -- 172.21.15.78:0/1219849557 shutdown_connections 2024-04-25T01:43:45.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.783+0000 7f0710cfc640 1 -- 172.21.15.78:0/1219849557 wait complete. 2024-04-25T01:43:45.785 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-04-25T01:43:44.899041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600231,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":145,"num_read_kb":124,"num_write":312,"num_write_kb":2993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":256,"ondisk_log_size":256,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360980,"kb_used_data":11808,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580844,"statfs":{"total":1151940427776,"available":1151570784256,"internally_reserved":0,"allocated":12091392,"data_stored":7576697,"data_compressed":283734,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5364,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":84,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003652"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158208+0000","last_change":"2024-04-25T01:43:17.423128+0000","last_active":"2024-04-25T01:43:23.158208+0000","last_peered":"2024-04-25T01:43:23.158208+0000","last_clean":"2024-04-25T01:43:23.158208+0000","last_became_active":"2024-04-25T01:43:17.422356+0000","last_became_peered":"2024-04-25T01:43:17.422356+0000","last_unstale":"2024-04-25T01:43:23.158208+0000","last_undegraded":"2024-04-25T01:43:23.158208+0000","last_fullsized":"2024-04-25T01:43:23.158208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:05.524727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157427+0000","last_change":"2024-04-25T01:43:17.422866+0000","last_active":"2024-04-25T01:43:23.157427+0000","last_peered":"2024-04-25T01:43:23.157427+0000","last_clean":"2024-04-25T01:43:23.157427+0000","last_became_active":"2024-04-25T01:43:17.422574+0000","last_became_peered":"2024-04-25T01:43:17.422574+0000","last_unstale":"2024-04-25T01:43:23.157427+0000","last_undegraded":"2024-04-25T01:43:23.157427+0000","last_fullsized":"2024-04-25T01:43:23.157427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.842361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157305+0000","last_change":"2024-04-25T01:43:17.419320+0000","last_active":"2024-04-25T01:43:23.157305+0000","last_peered":"2024-04-25T01:43:23.157305+0000","last_clean":"2024-04-25T01:43:23.157305+0000","last_became_active":"2024-04-25T01:43:17.418988+0000","last_became_peered":"2024-04-25T01:43:17.418988+0000","last_unstale":"2024-04-25T01:43:23.157305+0000","last_undegraded":"2024-04-25T01:43:23.157305+0000","last_fullsized":"2024-04-25T01:43:23.157305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:18.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158545+0000","last_change":"2024-04-25T01:43:17.430061+0000","last_active":"2024-04-25T01:43:23.158545+0000","last_peered":"2024-04-25T01:43:23.158545+0000","last_clean":"2024-04-25T01:43:23.158545+0000","last_became_active":"2024-04-25T01:43:17.429736+0000","last_became_peered":"2024-04-25T01:43:17.429736+0000","last_unstale":"2024-04-25T01:43:23.158545+0000","last_undegraded":"2024-04-25T01:43:23.158545+0000","last_fullsized":"2024-04-25T01:43:23.158545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:57.784221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721262+0000","last_change":"2024-04-25T01:43:17.422876+0000","last_active":"2024-04-25T01:43:22.721262+0000","last_peered":"2024-04-25T01:43:22.721262+0000","last_clean":"2024-04-25T01:43:22.721262+0000","last_became_active":"2024-04-25T01:43:17.422523+0000","last_became_peered":"2024-04-25T01:43:17.422523+0000","last_unstale":"2024-04-25T01:43:22.721262+0000","last_undegraded":"2024-04-25T01:43:22.721262+0000","last_fullsized":"2024-04-25T01:43:22.721262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:14.328021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.759026+0000","last_change":"2024-04-25T01:43:17.424941+0000","last_active":"2024-04-25T01:43:22.759026+0000","last_peered":"2024-04-25T01:43:22.759026+0000","last_clean":"2024-04-25T01:43:22.759026+0000","last_became_active":"2024-04-25T01:43:17.424620+0000","last_became_peered":"2024-04-25T01:43:17.424620+0000","last_unstale":"2024-04-25T01:43:22.759026+0000","last_undegraded":"2024-04-25T01:43:22.759026+0000","last_fullsized":"2024-04-25T01:43:22.759026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:06.111590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158253+0000","last_change":"2024-04-25T01:43:17.429188+0000","last_active":"2024-04-25T01:43:23.158253+0000","last_peered":"2024-04-25T01:43:23.158253+0000","last_clean":"2024-04-25T01:43:23.158253+0000","last_became_active":"2024-04-25T01:43:17.428837+0000","last_became_peered":"2024-04-25T01:43:17.428837+0000","last_unstale":"2024-04-25T01:43:23.158253+0000","last_undegraded":"2024-04-25T01:43:23.158253+0000","last_fullsized":"2024-04-25T01:43:23.158253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:08.923208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112431+0000","last_change":"2024-04-25T01:43:17.431402+0000","last_active":"2024-04-25T01:43:23.112431+0000","last_peered":"2024-04-25T01:43:23.112431+0000","last_clean":"2024-04-25T01:43:23.112431+0000","last_became_active":"2024-04-25T01:43:17.431044+0000","last_became_peered":"2024-04-25T01:43:17.431044+0000","last_unstale":"2024-04-25T01:43:23.112431+0000","last_undegraded":"2024-04-25T01:43:23.112431+0000","last_fullsized":"2024-04-25T01:43:23.112431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:20.731589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.430321+0000","last_change":"2024-04-25T01:43:17.430029+0000","last_active":"2024-04-25T01:43:22.430321+0000","last_peered":"2024-04-25T01:43:22.430321+0000","last_clean":"2024-04-25T01:43:22.430321+0000","last_became_active":"2024-04-25T01:43:17.428588+0000","last_became_peered":"2024-04-25T01:43:17.428588+0000","last_unstale":"2024-04-25T01:43:22.430321+0000","last_undegraded":"2024-04-25T01:43:22.430321+0000","last_fullsized":"2024-04-25T01:43:22.430321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:58.520175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.759364+0000","last_change":"2024-04-25T01:43:17.430377+0000","last_active":"2024-04-25T01:43:22.759364+0000","last_peered":"2024-04-25T01:43:22.759364+0000","last_clean":"2024-04-25T01:43:22.759364+0000","last_became_active":"2024-04-25T01:43:17.426609+0000","last_became_peered":"2024-04-25T01:43:17.426609+0000","last_unstale":"2024-04-25T01:43:22.759364+0000","last_undegraded":"2024-04-25T01:43:22.759364+0000","last_fullsized":"2024-04-25T01:43:22.759364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:10.696040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157957+0000","last_change":"2024-04-25T01:43:17.431561+0000","last_active":"2024-04-25T01:43:23.157957+0000","last_peered":"2024-04-25T01:43:23.157957+0000","last_clean":"2024-04-25T01:43:23.157957+0000","last_became_active":"2024-04-25T01:43:17.431229+0000","last_became_peered":"2024-04-25T01:43:17.431229+0000","last_unstale":"2024-04-25T01:43:23.157957+0000","last_undegraded":"2024-04-25T01:43:23.157957+0000","last_fullsized":"2024-04-25T01:43:23.157957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:03.686373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158407+0000","last_change":"2024-04-25T01:43:17.428442+0000","last_active":"2024-04-25T01:43:23.158407+0000","last_peered":"2024-04-25T01:43:23.158407+0000","last_clean":"2024-04-25T01:43:23.158407+0000","last_became_active":"2024-04-25T01:43:17.428052+0000","last_became_peered":"2024-04-25T01:43:17.428052+0000","last_unstale":"2024-04-25T01:43:23.158407+0000","last_undegraded":"2024-04-25T01:43:23.158407+0000","last_fullsized":"2024-04-25T01:43:23.158407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:06.097229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431003+0000","last_change":"2024-04-25T01:43:17.420520+0000","last_active":"2024-04-25T01:43:22.431003+0000","last_peered":"2024-04-25T01:43:22.431003+0000","last_clean":"2024-04-25T01:43:22.431003+0000","last_became_active":"2024-04-25T01:43:17.420171+0000","last_became_peered":"2024-04-25T01:43:17.420171+0000","last_unstale":"2024-04-25T01:43:22.431003+0000","last_undegraded":"2024-04-25T01:43:22.431003+0000","last_fullsized":"2024-04-25T01:43:22.431003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:43.699472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.156824+0000","last_change":"2024-04-25T01:43:17.422114+0000","last_active":"2024-04-25T01:43:23.156824+0000","last_peered":"2024-04-25T01:43:23.156824+0000","last_clean":"2024-04-25T01:43:23.156824+0000","last_became_active":"2024-04-25T01:43:17.421729+0000","last_became_peered":"2024-04-25T01:43:17.421729+0000","last_unstale":"2024-04-25T01:43:23.156824+0000","last_undegraded":"2024-04-25T01:43:23.156824+0000","last_fullsized":"2024-04-25T01:43:23.156824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:00.955123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.156797+0000","last_change":"2024-04-25T01:43:17.427344+0000","last_active":"2024-04-25T01:43:23.156797+0000","last_peered":"2024-04-25T01:43:23.156797+0000","last_clean":"2024-04-25T01:43:23.156797+0000","last_became_active":"2024-04-25T01:43:17.427044+0000","last_became_peered":"2024-04-25T01:43:17.427044+0000","last_unstale":"2024-04-25T01:43:23.156797+0000","last_undegraded":"2024-04-25T01:43:23.156797+0000","last_fullsized":"2024-04-25T01:43:23.156797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:24.564654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.429595+0000","last_change":"2024-04-25T01:43:17.427604+0000","last_active":"2024-04-25T01:43:22.429595+0000","last_peered":"2024-04-25T01:43:22.429595+0000","last_clean":"2024-04-25T01:43:22.429595+0000","last_became_active":"2024-04-25T01:43:17.427286+0000","last_became_peered":"2024-04-25T01:43:17.427286+0000","last_unstale":"2024-04-25T01:43:22.429595+0000","last_undegraded":"2024-04-25T01:43:22.429595+0000","last_fullsized":"2024-04-25T01:43:22.429595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:25.584150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158017+0000","last_change":"2024-04-25T01:43:17.423401+0000","last_active":"2024-04-25T01:43:23.158017+0000","last_peered":"2024-04-25T01:43:23.158017+0000","last_clean":"2024-04-25T01:43:23.158017+0000","last_became_active":"2024-04-25T01:43:17.422807+0000","last_became_peered":"2024-04-25T01:43:17.422807+0000","last_unstale":"2024-04-25T01:43:23.158017+0000","last_undegraded":"2024-04-25T01:43:23.158017+0000","last_fullsized":"2024-04-25T01:43:23.158017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:58.461242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157769+0000","last_change":"2024-04-25T01:43:17.422054+0000","last_active":"2024-04-25T01:43:23.157769+0000","last_peered":"2024-04-25T01:43:23.157769+0000","last_clean":"2024-04-25T01:43:23.157769+0000","last_became_active":"2024-04-25T01:43:17.421746+0000","last_became_peered":"2024-04-25T01:43:17.421746+0000","last_unstale":"2024-04-25T01:43:23.157769+0000","last_undegraded":"2024-04-25T01:43:23.157769+0000","last_fullsized":"2024-04-25T01:43:23.157769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:28.405852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158207+0000","last_change":"2024-04-25T01:43:17.429288+0000","last_active":"2024-04-25T01:43:23.158207+0000","last_peered":"2024-04-25T01:43:23.158207+0000","last_clean":"2024-04-25T01:43:23.158207+0000","last_became_active":"2024-04-25T01:43:17.427794+0000","last_became_peered":"2024-04-25T01:43:17.427794+0000","last_unstale":"2024-04-25T01:43:23.158207+0000","last_undegraded":"2024-04-25T01:43:23.158207+0000","last_fullsized":"2024-04-25T01:43:23.158207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:33.003361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157921+0000","last_change":"2024-04-25T01:43:21.438427+0000","last_active":"2024-04-25T01:43:23.157921+0000","last_peered":"2024-04-25T01:43:23.157921+0000","last_clean":"2024-04-25T01:43:23.157921+0000","last_became_active":"2024-04-25T01:43:21.438018+0000","last_became_peered":"2024-04-25T01:43:21.438018+0000","last_unstale":"2024-04-25T01:43:23.157921+0000","last_undegraded":"2024-04-25T01:43:23.157921+0000","last_fullsized":"2024-04-25T01:43:23.157921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:46.509437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157946+0000","last_change":"2024-04-25T01:43:17.031110+0000","last_active":"2024-04-25T01:43:23.157946+0000","last_peered":"2024-04-25T01:43:23.157946+0000","last_clean":"2024-04-25T01:43:23.157946+0000","last_became_active":"2024-04-25T01:43:17.030806+0000","last_became_peered":"2024-04-25T01:43:17.030806+0000","last_unstale":"2024-04-25T01:43:23.157946+0000","last_undegraded":"2024-04-25T01:43:23.157946+0000","last_fullsized":"2024-04-25T01:43:23.157946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:52:12.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112942+0000","last_change":"2024-04-25T01:43:17.428847+0000","last_active":"2024-04-25T01:43:23.112942+0000","last_peered":"2024-04-25T01:43:23.112942+0000","last_clean":"2024-04-25T01:43:23.112942+0000","last_became_active":"2024-04-25T01:43:17.428553+0000","last_became_peered":"2024-04-25T01:43:17.428553+0000","last_unstale":"2024-04-25T01:43:23.112942+0000","last_undegraded":"2024-04-25T01:43:23.112942+0000","last_fullsized":"2024-04-25T01:43:23.112942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:13.973275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157346+0000","last_change":"2024-04-25T01:43:21.452786+0000","last_active":"2024-04-25T01:43:23.157346+0000","last_peered":"2024-04-25T01:43:23.157346+0000","last_clean":"2024-04-25T01:43:23.157346+0000","last_became_active":"2024-04-25T01:43:21.451595+0000","last_became_peered":"2024-04-25T01:43:21.451595+0000","last_unstale":"2024-04-25T01:43:23.157346+0000","last_undegraded":"2024-04-25T01:43:23.157346+0000","last_fullsized":"2024-04-25T01:43:23.157346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:06.559390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.113136+0000","last_change":"2024-04-25T01:43:17.032954+0000","last_active":"2024-04-25T01:43:23.113136+0000","last_peered":"2024-04-25T01:43:23.113136+0000","last_clean":"2024-04-25T01:43:23.113136+0000","last_became_active":"2024-04-25T01:43:17.032627+0000","last_became_peered":"2024-04-25T01:43:17.032627+0000","last_unstale":"2024-04-25T01:43:23.113136+0000","last_undegraded":"2024-04-25T01:43:23.113136+0000","last_fullsized":"2024-04-25T01:43:23.113136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:07.167227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157128+0000","last_change":"2024-04-25T01:43:17.419451+0000","last_active":"2024-04-25T01:43:23.157128+0000","last_peered":"2024-04-25T01:43:23.157128+0000","last_clean":"2024-04-25T01:43:23.157128+0000","last_became_active":"2024-04-25T01:43:17.419111+0000","last_became_peered":"2024-04-25T01:43:17.419111+0000","last_unstale":"2024-04-25T01:43:23.157128+0000","last_undegraded":"2024-04-25T01:43:23.157128+0000","last_fullsized":"2024-04-25T01:43:23.157128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:41.284248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157863+0000","last_change":"2024-04-25T01:43:21.452525+0000","last_active":"2024-04-25T01:43:23.157863+0000","last_peered":"2024-04-25T01:43:23.157863+0000","last_clean":"2024-04-25T01:43:23.157863+0000","last_became_active":"2024-04-25T01:43:21.451834+0000","last_became_peered":"2024-04-25T01:43:21.451834+0000","last_unstale":"2024-04-25T01:43:23.157863+0000","last_undegraded":"2024-04-25T01:43:23.157863+0000","last_fullsized":"2024-04-25T01:43:23.157863+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:34.866314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721753+0000","last_change":"2024-04-25T01:43:17.035737+0000","last_active":"2024-04-25T01:43:22.721753+0000","last_peered":"2024-04-25T01:43:22.721753+0000","last_clean":"2024-04-25T01:43:22.721753+0000","last_became_active":"2024-04-25T01:43:17.034709+0000","last_became_peered":"2024-04-25T01:43:17.034709+0000","last_unstale":"2024-04-25T01:43:22.721753+0000","last_undegraded":"2024-04-25T01:43:22.721753+0000","last_fullsized":"2024-04-25T01:43:22.721753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:50.419193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.429986+0000","last_change":"2024-04-25T01:43:17.422530+0000","last_active":"2024-04-25T01:43:22.429986+0000","last_peered":"2024-04-25T01:43:22.429986+0000","last_clean":"2024-04-25T01:43:22.429986+0000","last_became_active":"2024-04-25T01:43:17.422197+0000","last_became_peered":"2024-04-25T01:43:17.422197+0000","last_unstale":"2024-04-25T01:43:22.429986+0000","last_undegraded":"2024-04-25T01:43:22.429986+0000","last_fullsized":"2024-04-25T01:43:22.429986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:25.250947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157384+0000","last_change":"2024-04-25T01:43:21.446869+0000","last_active":"2024-04-25T01:43:23.157384+0000","last_peered":"2024-04-25T01:43:23.157384+0000","last_clean":"2024-04-25T01:43:23.157384+0000","last_became_active":"2024-04-25T01:43:21.446446+0000","last_became_peered":"2024-04-25T01:43:21.446446+0000","last_unstale":"2024-04-25T01:43:23.157384+0000","last_undegraded":"2024-04-25T01:43:23.157384+0000","last_fullsized":"2024-04-25T01:43:23.157384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:21.023845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721751+0000","last_change":"2024-04-25T01:43:17.027614+0000","last_active":"2024-04-25T01:43:22.721751+0000","last_peered":"2024-04-25T01:43:22.721751+0000","last_clean":"2024-04-25T01:43:22.721751+0000","last_became_active":"2024-04-25T01:43:17.027235+0000","last_became_peered":"2024-04-25T01:43:17.027235+0000","last_unstale":"2024-04-25T01:43:22.721751+0000","last_undegraded":"2024-04-25T01:43:22.721751+0000","last_fullsized":"2024-04-25T01:43:22.721751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:45.627146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431928+0000","last_change":"2024-04-25T01:43:17.425813+0000","last_active":"2024-04-25T01:43:22.431928+0000","last_peered":"2024-04-25T01:43:22.431928+0000","last_clean":"2024-04-25T01:43:22.431928+0000","last_became_active":"2024-04-25T01:43:17.425469+0000","last_became_peered":"2024-04-25T01:43:17.425469+0000","last_unstale":"2024-04-25T01:43:22.431928+0000","last_undegraded":"2024-04-25T01:43:22.431928+0000","last_fullsized":"2024-04-25T01:43:22.431928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:49.863358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158138+0000","last_change":"2024-04-25T01:43:21.444557+0000","last_active":"2024-04-25T01:43:23.158138+0000","last_peered":"2024-04-25T01:43:23.158138+0000","last_clean":"2024-04-25T01:43:23.158138+0000","last_became_active":"2024-04-25T01:43:21.443809+0000","last_became_peered":"2024-04-25T01:43:21.443809+0000","last_unstale":"2024-04-25T01:43:23.158138+0000","last_undegraded":"2024-04-25T01:43:23.158138+0000","last_fullsized":"2024-04-25T01:43:23.158138+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:29.488448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:43.419368+0000","last_change":"2024-04-25T01:43:17.032280+0000","last_active":"2024-04-25T01:43:43.419368+0000","last_peered":"2024-04-25T01:43:43.419368+0000","last_clean":"2024-04-25T01:43:43.419368+0000","last_became_active":"2024-04-25T01:43:17.031649+0000","last_became_peered":"2024-04-25T01:43:17.031649+0000","last_unstale":"2024-04-25T01:43:43.419368+0000","last_undegraded":"2024-04-25T01:43:43.419368+0000","last_fullsized":"2024-04-25T01:43:43.419368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:08.532261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721299+0000","last_change":"2024-04-25T01:43:17.420350+0000","last_active":"2024-04-25T01:43:22.721299+0000","last_peered":"2024-04-25T01:43:22.721299+0000","last_clean":"2024-04-25T01:43:22.721299+0000","last_became_active":"2024-04-25T01:43:17.420024+0000","last_became_peered":"2024-04-25T01:43:17.420024+0000","last_unstale":"2024-04-25T01:43:22.721299+0000","last_undegraded":"2024-04-25T01:43:22.721299+0000","last_fullsized":"2024-04-25T01:43:22.721299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:19.395939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158271+0000","last_change":"2024-04-25T01:43:21.442683+0000","last_active":"2024-04-25T01:43:23.158271+0000","last_peered":"2024-04-25T01:43:23.158271+0000","last_clean":"2024-04-25T01:43:23.158271+0000","last_became_active":"2024-04-25T01:43:21.442222+0000","last_became_peered":"2024-04-25T01:43:21.442222+0000","last_unstale":"2024-04-25T01:43:23.158271+0000","last_undegraded":"2024-04-25T01:43:23.158271+0000","last_fullsized":"2024-04-25T01:43:23.158271+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:25.196600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.758997+0000","last_change":"2024-04-25T01:43:17.028054+0000","last_active":"2024-04-25T01:43:22.758997+0000","last_peered":"2024-04-25T01:43:22.758997+0000","last_clean":"2024-04-25T01:43:22.758997+0000","last_became_active":"2024-04-25T01:43:17.027639+0000","last_became_peered":"2024-04-25T01:43:17.027639+0000","last_unstale":"2024-04-25T01:43:22.758997+0000","last_undegraded":"2024-04-25T01:43:22.758997+0000","last_fullsized":"2024-04-25T01:43:22.758997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:40.569221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431758+0000","last_change":"2024-04-25T01:43:17.430417+0000","last_active":"2024-04-25T01:43:22.431758+0000","last_peered":"2024-04-25T01:43:22.431758+0000","last_clean":"2024-04-25T01:43:22.431758+0000","last_became_active":"2024-04-25T01:43:17.430024+0000","last_became_peered":"2024-04-25T01:43:17.430024+0000","last_unstale":"2024-04-25T01:43:22.431758+0000","last_undegraded":"2024-04-25T01:43:22.431758+0000","last_fullsized":"2024-04-25T01:43:22.431758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:56.834530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158064+0000","last_change":"2024-04-25T01:43:21.447732+0000","last_active":"2024-04-25T01:43:23.158064+0000","last_peered":"2024-04-25T01:43:23.158064+0000","last_clean":"2024-04-25T01:43:23.158064+0000","last_became_active":"2024-04-25T01:43:21.447315+0000","last_became_peered":"2024-04-25T01:43:21.447315+0000","last_unstale":"2024-04-25T01:43:23.158064+0000","last_undegraded":"2024-04-25T01:43:23.158064+0000","last_fullsized":"2024-04-25T01:43:23.158064+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:18.138828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.008657+0000","last_change":"2024-04-25T01:43:17.018769+0000","last_active":"2024-04-25T01:43:42.008657+0000","last_peered":"2024-04-25T01:43:42.008657+0000","last_clean":"2024-04-25T01:43:42.008657+0000","last_became_active":"2024-04-25T01:43:17.018169+0000","last_became_peered":"2024-04-25T01:43:17.018169+0000","last_unstale":"2024-04-25T01:43:42.008657+0000","last_undegraded":"2024-04-25T01:43:42.008657+0000","last_fullsized":"2024-04-25T01:43:42.008657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:59.112199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112568+0000","last_change":"2024-04-25T01:43:17.426289+0000","last_active":"2024-04-25T01:43:23.112568+0000","last_peered":"2024-04-25T01:43:23.112568+0000","last_clean":"2024-04-25T01:43:23.112568+0000","last_became_active":"2024-04-25T01:43:17.425512+0000","last_became_peered":"2024-04-25T01:43:17.425512+0000","last_unstale":"2024-04-25T01:43:23.112568+0000","last_undegraded":"2024-04-25T01:43:23.112568+0000","last_fullsized":"2024-04-25T01:43:23.112568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:47.956758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157217+0000","last_change":"2024-04-25T01:43:21.449131+0000","last_active":"2024-04-25T01:43:23.157217+0000","last_peered":"2024-04-25T01:43:23.157217+0000","last_clean":"2024-04-25T01:43:23.157217+0000","last_became_active":"2024-04-25T01:43:21.448752+0000","last_became_peered":"2024-04-25T01:43:21.448752+0000","last_unstale":"2024-04-25T01:43:23.157217+0000","last_undegraded":"2024-04-25T01:43:23.157217+0000","last_fullsized":"2024-04-25T01:43:23.157217+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:23.109582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.425543+0000","last_change":"2024-04-25T01:43:17.032414+0000","last_active":"2024-04-25T01:43:42.425543+0000","last_peered":"2024-04-25T01:43:42.425543+0000","last_clean":"2024-04-25T01:43:42.425543+0000","last_became_active":"2024-04-25T01:43:17.031848+0000","last_became_peered":"2024-04-25T01:43:17.031848+0000","last_unstale":"2024-04-25T01:43:42.425543+0000","last_undegraded":"2024-04-25T01:43:42.425543+0000","last_fullsized":"2024-04-25T01:43:42.425543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:37.034113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157484+0000","last_change":"2024-04-25T01:43:17.422831+0000","last_active":"2024-04-25T01:43:23.157484+0000","last_peered":"2024-04-25T01:43:23.157484+0000","last_clean":"2024-04-25T01:43:23.157484+0000","last_became_active":"2024-04-25T01:43:17.422238+0000","last_became_peered":"2024-04-25T01:43:17.422238+0000","last_unstale":"2024-04-25T01:43:23.157484+0000","last_undegraded":"2024-04-25T01:43:23.157484+0000","last_fullsized":"2024-04-25T01:43:23.157484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:05.740054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112547+0000","last_change":"2024-04-25T01:43:21.442268+0000","last_active":"2024-04-25T01:43:23.112547+0000","last_peered":"2024-04-25T01:43:23.112547+0000","last_clean":"2024-04-25T01:43:23.112547+0000","last_became_active":"2024-04-25T01:43:21.441844+0000","last_became_peered":"2024-04-25T01:43:21.441844+0000","last_unstale":"2024-04-25T01:43:23.112547+0000","last_undegraded":"2024-04-25T01:43:23.112547+0000","last_fullsized":"2024-04-25T01:43:23.112547+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:21.627312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:41.001380+0000","last_change":"2024-04-25T01:43:17.022576+0000","last_active":"2024-04-25T01:43:41.001380+0000","last_peered":"2024-04-25T01:43:41.001380+0000","last_clean":"2024-04-25T01:43:41.001380+0000","last_became_active":"2024-04-25T01:43:17.022223+0000","last_became_peered":"2024-04-25T01:43:17.022223+0000","last_unstale":"2024-04-25T01:43:41.001380+0000","last_undegraded":"2024-04-25T01:43:41.001380+0000","last_fullsized":"2024-04-25T01:43:41.001380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:23.905397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.430248+0000","last_change":"2024-04-25T01:43:17.421460+0000","last_active":"2024-04-25T01:43:22.430248+0000","last_peered":"2024-04-25T01:43:22.430248+0000","last_clean":"2024-04-25T01:43:22.430248+0000","last_became_active":"2024-04-25T01:43:17.421153+0000","last_became_peered":"2024-04-25T01:43:17.421153+0000","last_unstale":"2024-04-25T01:43:22.430248+0000","last_undegraded":"2024-04-25T01:43:22.430248+0000","last_fullsized":"2024-04-25T01:43:22.430248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:50.285476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.156968+0000","last_change":"2024-04-25T01:43:21.450961+0000","last_active":"2024-04-25T01:43:23.156968+0000","last_peered":"2024-04-25T01:43:23.156968+0000","last_clean":"2024-04-25T01:43:23.156968+0000","last_became_active":"2024-04-25T01:43:21.450578+0000","last_became_peered":"2024-04-25T01:43:21.450578+0000","last_unstale":"2024-04-25T01:43:23.156968+0000","last_undegraded":"2024-04-25T01:43:23.156968+0000","last_fullsized":"2024-04-25T01:43:23.156968+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:04.222443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:41.001075+0000","last_change":"2024-04-25T01:43:17.030973+0000","last_active":"2024-04-25T01:43:41.001075+0000","last_peered":"2024-04-25T01:43:41.001075+0000","last_clean":"2024-04-25T01:43:41.001075+0000","last_became_active":"2024-04-25T01:43:17.030458+0000","last_became_peered":"2024-04-25T01:43:17.030458+0000","last_unstale":"2024-04-25T01:43:41.001075+0000","last_undegraded":"2024-04-25T01:43:41.001075+0000","last_fullsized":"2024-04-25T01:43:41.001075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:19.187667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157097+0000","last_change":"2024-04-25T01:43:17.426748+0000","last_active":"2024-04-25T01:43:23.157097+0000","last_peered":"2024-04-25T01:43:23.157097+0000","last_clean":"2024-04-25T01:43:23.157097+0000","last_became_active":"2024-04-25T01:43:17.426447+0000","last_became_peered":"2024-04-25T01:43:17.426447+0000","last_unstale":"2024-04-25T01:43:23.157097+0000","last_undegraded":"2024-04-25T01:43:23.157097+0000","last_fullsized":"2024-04-25T01:43:23.157097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:39.794142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158014+0000","last_change":"2024-04-25T01:43:21.452287+0000","last_active":"2024-04-25T01:43:23.158014+0000","last_peered":"2024-04-25T01:43:23.158014+0000","last_clean":"2024-04-25T01:43:23.158014+0000","last_became_active":"2024-04-25T01:43:21.451765+0000","last_became_peered":"2024-04-25T01:43:21.451765+0000","last_unstale":"2024-04-25T01:43:23.158014+0000","last_undegraded":"2024-04-25T01:43:23.158014+0000","last_fullsized":"2024-04-25T01:43:23.158014+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:02.353645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.417131+0000","last_change":"2024-04-25T01:43:17.022319+0000","last_active":"2024-04-25T01:43:42.417131+0000","last_peered":"2024-04-25T01:43:42.417131+0000","last_clean":"2024-04-25T01:43:42.417131+0000","last_became_active":"2024-04-25T01:43:17.021944+0000","last_became_peered":"2024-04-25T01:43:17.021944+0000","last_unstale":"2024-04-25T01:43:42.417131+0000","last_undegraded":"2024-04-25T01:43:42.417131+0000","last_fullsized":"2024-04-25T01:43:42.417131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:42.856997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.430184+0000","last_change":"2024-04-25T01:43:17.418164+0000","last_active":"2024-04-25T01:43:22.430184+0000","last_peered":"2024-04-25T01:43:22.430184+0000","last_clean":"2024-04-25T01:43:22.430184+0000","last_became_active":"2024-04-25T01:43:17.417846+0000","last_became_peered":"2024-04-25T01:43:17.417846+0000","last_unstale":"2024-04-25T01:43:22.430184+0000","last_undegraded":"2024-04-25T01:43:22.430184+0000","last_fullsized":"2024-04-25T01:43:22.430184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:25.973457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157297+0000","last_change":"2024-04-25T01:43:21.443429+0000","last_active":"2024-04-25T01:43:23.157297+0000","last_peered":"2024-04-25T01:43:23.157297+0000","last_clean":"2024-04-25T01:43:23.157297+0000","last_became_active":"2024-04-25T01:43:21.442210+0000","last_became_peered":"2024-04-25T01:43:21.442210+0000","last_unstale":"2024-04-25T01:43:23.157297+0000","last_undegraded":"2024-04-25T01:43:23.157297+0000","last_fullsized":"2024-04-25T01:43:23.157297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:29.578740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'11","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:41.000319+0000","last_change":"2024-04-25T01:43:17.020501+0000","last_active":"2024-04-25T01:43:41.000319+0000","last_peered":"2024-04-25T01:43:41.000319+0000","last_clean":"2024-04-25T01:43:41.000319+0000","last_became_active":"2024-04-25T01:43:17.020160+0000","last_became_peered":"2024-04-25T01:43:17.020160+0000","last_unstale":"2024-04-25T01:43:41.000319+0000","last_undegraded":"2024-04-25T01:43:41.000319+0000","last_fullsized":"2024-04-25T01:43:41.000319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:49.273381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721533+0000","last_change":"2024-04-25T01:43:17.426895+0000","last_active":"2024-04-25T01:43:22.721533+0000","last_peered":"2024-04-25T01:43:22.721533+0000","last_clean":"2024-04-25T01:43:22.721533+0000","last_became_active":"2024-04-25T01:43:17.426572+0000","last_became_peered":"2024-04-25T01:43:17.426572+0000","last_unstale":"2024-04-25T01:43:22.721533+0000","last_undegraded":"2024-04-25T01:43:22.721533+0000","last_fullsized":"2024-04-25T01:43:22.721533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:36.166282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.432315+0000","last_change":"2024-04-25T01:43:21.442204+0000","last_active":"2024-04-25T01:43:22.432315+0000","last_peered":"2024-04-25T01:43:22.432315+0000","last_clean":"2024-04-25T01:43:22.432315+0000","last_became_active":"2024-04-25T01:43:21.441779+0000","last_became_peered":"2024-04-25T01:43:21.441779+0000","last_unstale":"2024-04-25T01:43:22.432315+0000","last_undegraded":"2024-04-25T01:43:22.432315+0000","last_fullsized":"2024-04-25T01:43:22.432315+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:12.075657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:43.442417+0000","last_change":"2024-04-25T01:43:17.028737+0000","last_active":"2024-04-25T01:43:43.442417+0000","last_peered":"2024-04-25T01:43:43.442417+0000","last_clean":"2024-04-25T01:43:43.442417+0000","last_became_active":"2024-04-25T01:43:17.028349+0000","last_became_peered":"2024-04-25T01:43:17.028349+0000","last_unstale":"2024-04-25T01:43:43.442417+0000","last_undegraded":"2024-04-25T01:43:43.442417+0000","last_fullsized":"2024-04-25T01:43:43.442417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:04.802232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431051+0000","last_change":"2024-04-25T01:43:17.436315+0000","last_active":"2024-04-25T01:43:22.431051+0000","last_peered":"2024-04-25T01:43:22.431051+0000","last_clean":"2024-04-25T01:43:22.431051+0000","last_became_active":"2024-04-25T01:43:17.436007+0000","last_became_peered":"2024-04-25T01:43:17.436007+0000","last_unstale":"2024-04-25T01:43:22.431051+0000","last_undegraded":"2024-04-25T01:43:22.431051+0000","last_fullsized":"2024-04-25T01:43:22.431051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:49.091092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157664+0000","last_change":"2024-04-25T01:43:21.446834+0000","last_active":"2024-04-25T01:43:23.157664+0000","last_peered":"2024-04-25T01:43:23.157664+0000","last_clean":"2024-04-25T01:43:23.157664+0000","last_became_active":"2024-04-25T01:43:21.446346+0000","last_became_peered":"2024-04-25T01:43:21.446346+0000","last_unstale":"2024-04-25T01:43:23.157664+0000","last_undegraded":"2024-04-25T01:43:23.157664+0000","last_fullsized":"2024-04-25T01:43:23.157664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:00.050981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.425322+0000","last_change":"2024-04-25T01:43:17.029513+0000","last_active":"2024-04-25T01:43:42.425322+0000","last_peered":"2024-04-25T01:43:42.425322+0000","last_clean":"2024-04-25T01:43:42.425322+0000","last_became_active":"2024-04-25T01:43:17.029203+0000","last_became_peered":"2024-04-25T01:43:17.029203+0000","last_unstale":"2024-04-25T01:43:42.425322+0000","last_undegraded":"2024-04-25T01:43:42.425322+0000","last_fullsized":"2024-04-25T01:43:42.425322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:47.002587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158090+0000","last_change":"2024-04-25T01:43:17.435273+0000","last_active":"2024-04-25T01:43:23.158090+0000","last_peered":"2024-04-25T01:43:23.158090+0000","last_clean":"2024-04-25T01:43:23.158090+0000","last_became_active":"2024-04-25T01:43:17.434892+0000","last_became_peered":"2024-04-25T01:43:17.434892+0000","last_unstale":"2024-04-25T01:43:23.158090+0000","last_undegraded":"2024-04-25T01:43:23.158090+0000","last_fullsized":"2024-04-25T01:43:23.158090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:55.342440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157761+0000","last_change":"2024-04-25T01:43:21.448363+0000","last_active":"2024-04-25T01:43:23.157761+0000","last_peered":"2024-04-25T01:43:23.157761+0000","last_clean":"2024-04-25T01:43:23.157761+0000","last_became_active":"2024-04-25T01:43:21.445980+0000","last_became_peered":"2024-04-25T01:43:21.445980+0000","last_unstale":"2024-04-25T01:43:23.157761+0000","last_undegraded":"2024-04-25T01:43:23.157761+0000","last_fullsized":"2024-04-25T01:43:23.157761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:05.890579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:43.422804+0000","last_change":"2024-04-25T01:43:17.028557+0000","last_active":"2024-04-25T01:43:43.422804+0000","last_peered":"2024-04-25T01:43:43.422804+0000","last_clean":"2024-04-25T01:43:43.422804+0000","last_became_active":"2024-04-25T01:43:17.028199+0000","last_became_peered":"2024-04-25T01:43:17.028199+0000","last_unstale":"2024-04-25T01:43:43.422804+0000","last_undegraded":"2024-04-25T01:43:43.422804+0000","last_fullsized":"2024-04-25T01:43:43.422804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:41.809220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157293+0000","last_change":"2024-04-25T01:43:17.437002+0000","last_active":"2024-04-25T01:43:23.157293+0000","last_peered":"2024-04-25T01:43:23.157293+0000","last_clean":"2024-04-25T01:43:23.157293+0000","last_became_active":"2024-04-25T01:43:17.436509+0000","last_became_peered":"2024-04-25T01:43:17.436509+0000","last_unstale":"2024-04-25T01:43:23.157293+0000","last_undegraded":"2024-04-25T01:43:23.157293+0000","last_fullsized":"2024-04-25T01:43:23.157293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:49.329249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431432+0000","last_change":"2024-04-25T01:43:21.447748+0000","last_active":"2024-04-25T01:43:22.431432+0000","last_peered":"2024-04-25T01:43:22.431432+0000","last_clean":"2024-04-25T01:43:22.431432+0000","last_became_active":"2024-04-25T01:43:21.447368+0000","last_became_peered":"2024-04-25T01:43:21.447368+0000","last_unstale":"2024-04-25T01:43:22.431432+0000","last_undegraded":"2024-04-25T01:43:22.431432+0000","last_fullsized":"2024-04-25T01:43:22.431432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:53.506298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721191+0000","last_change":"2024-04-25T01:43:17.028185+0000","last_active":"2024-04-25T01:43:22.721191+0000","last_peered":"2024-04-25T01:43:22.721191+0000","last_clean":"2024-04-25T01:43:22.721191+0000","last_became_active":"2024-04-25T01:43:17.027851+0000","last_became_peered":"2024-04-25T01:43:17.027851+0000","last_unstale":"2024-04-25T01:43:22.721191+0000","last_undegraded":"2024-04-25T01:43:22.721191+0000","last_fullsized":"2024-04-25T01:43:22.721191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:02.320793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431342+0000","last_change":"2024-04-25T01:43:17.424990+0000","last_active":"2024-04-25T01:43:22.431342+0000","last_peered":"2024-04-25T01:43:22.431342+0000","last_clean":"2024-04-25T01:43:22.431342+0000","last_became_active":"2024-04-25T01:43:17.424598+0000","last_became_peered":"2024-04-25T01:43:17.424598+0000","last_unstale":"2024-04-25T01:43:22.431342+0000","last_undegraded":"2024-04-25T01:43:22.431342+0000","last_fullsized":"2024-04-25T01:43:22.431342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:34.855089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431511+0000","last_change":"2024-04-25T01:43:21.444138+0000","last_active":"2024-04-25T01:43:22.431511+0000","last_peered":"2024-04-25T01:43:22.431511+0000","last_clean":"2024-04-25T01:43:22.431511+0000","last_became_active":"2024-04-25T01:43:21.443283+0000","last_became_peered":"2024-04-25T01:43:21.443283+0000","last_unstale":"2024-04-25T01:43:22.431511+0000","last_undegraded":"2024-04-25T01:43:22.431511+0000","last_fullsized":"2024-04-25T01:43:22.431511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:46.275789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721617+0000","last_change":"2024-04-25T01:43:17.027892+0000","last_active":"2024-04-25T01:43:22.721617+0000","last_peered":"2024-04-25T01:43:22.721617+0000","last_clean":"2024-04-25T01:43:22.721617+0000","last_became_active":"2024-04-25T01:43:17.027476+0000","last_became_peered":"2024-04-25T01:43:17.027476+0000","last_unstale":"2024-04-25T01:43:22.721617+0000","last_undegraded":"2024-04-25T01:43:22.721617+0000","last_fullsized":"2024-04-25T01:43:22.721617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:31.079700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157897+0000","last_change":"2024-04-25T01:43:17.431491+0000","last_active":"2024-04-25T01:43:23.157897+0000","last_peered":"2024-04-25T01:43:23.157897+0000","last_clean":"2024-04-25T01:43:23.157897+0000","last_became_active":"2024-04-25T01:43:17.431160+0000","last_became_peered":"2024-04-25T01:43:17.431160+0000","last_unstale":"2024-04-25T01:43:23.157897+0000","last_undegraded":"2024-04-25T01:43:23.157897+0000","last_fullsized":"2024-04-25T01:43:23.157897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:03.612857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721409+0000","last_change":"2024-04-25T01:43:21.626398+0000","last_active":"2024-04-25T01:43:22.721409+0000","last_peered":"2024-04-25T01:43:22.721409+0000","last_clean":"2024-04-25T01:43:22.721409+0000","last_became_active":"2024-04-25T01:43:21.625856+0000","last_became_peered":"2024-04-25T01:43:21.625856+0000","last_unstale":"2024-04-25T01:43:22.721409+0000","last_undegraded":"2024-04-25T01:43:22.721409+0000","last_fullsized":"2024-04-25T01:43:22.721409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:53.000912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157615+0000","last_change":"2024-04-25T01:43:17.027564+0000","last_active":"2024-04-25T01:43:23.157615+0000","last_peered":"2024-04-25T01:43:23.157615+0000","last_clean":"2024-04-25T01:43:23.157615+0000","last_became_active":"2024-04-25T01:43:17.027218+0000","last_became_peered":"2024-04-25T01:43:17.027218+0000","last_unstale":"2024-04-25T01:43:23.157615+0000","last_undegraded":"2024-04-25T01:43:23.157615+0000","last_fullsized":"2024-04-25T01:43:23.157615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:52.658023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.759150+0000","last_change":"2024-04-25T01:43:17.431647+0000","last_active":"2024-04-25T01:43:22.759150+0000","last_peered":"2024-04-25T01:43:22.759150+0000","last_clean":"2024-04-25T01:43:22.759150+0000","last_became_active":"2024-04-25T01:43:17.431048+0000","last_became_peered":"2024-04-25T01:43:17.431048+0000","last_unstale":"2024-04-25T01:43:22.759150+0000","last_undegraded":"2024-04-25T01:43:22.759150+0000","last_fullsized":"2024-04-25T01:43:22.759150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:30.124585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431037+0000","last_change":"2024-04-25T01:43:21.450326+0000","last_active":"2024-04-25T01:43:22.431037+0000","last_peered":"2024-04-25T01:43:22.431037+0000","last_clean":"2024-04-25T01:43:22.431037+0000","last_became_active":"2024-04-25T01:43:21.449925+0000","last_became_peered":"2024-04-25T01:43:21.449925+0000","last_unstale":"2024-04-25T01:43:22.431037+0000","last_undegraded":"2024-04-25T01:43:22.431037+0000","last_fullsized":"2024-04-25T01:43:22.431037+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:39.896905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157771+0000","last_change":"2024-04-25T01:43:17.032223+0000","last_active":"2024-04-25T01:43:23.157771+0000","last_peered":"2024-04-25T01:43:23.157771+0000","last_clean":"2024-04-25T01:43:23.157771+0000","last_became_active":"2024-04-25T01:43:17.031774+0000","last_became_peered":"2024-04-25T01:43:17.031774+0000","last_unstale":"2024-04-25T01:43:23.157771+0000","last_undegraded":"2024-04-25T01:43:23.157771+0000","last_fullsized":"2024-04-25T01:43:23.157771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:03.730850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.759222+0000","last_change":"2024-04-25T01:43:17.425756+0000","last_active":"2024-04-25T01:43:22.759222+0000","last_peered":"2024-04-25T01:43:22.759222+0000","last_clean":"2024-04-25T01:43:22.759222+0000","last_became_active":"2024-04-25T01:43:17.425433+0000","last_became_peered":"2024-04-25T01:43:17.425433+0000","last_unstale":"2024-04-25T01:43:22.759222+0000","last_undegraded":"2024-04-25T01:43:22.759222+0000","last_fullsized":"2024-04-25T01:43:22.759222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:29.776771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431289+0000","last_change":"2024-04-25T01:43:21.625898+0000","last_active":"2024-04-25T01:43:22.431289+0000","last_peered":"2024-04-25T01:43:22.431289+0000","last_clean":"2024-04-25T01:43:22.431289+0000","last_became_active":"2024-04-25T01:43:21.625460+0000","last_became_peered":"2024-04-25T01:43:21.625460+0000","last_unstale":"2024-04-25T01:43:22.431289+0000","last_undegraded":"2024-04-25T01:43:22.431289+0000","last_fullsized":"2024-04-25T01:43:22.431289+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:31.199172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431720+0000","last_change":"2024-04-25T01:43:17.424709+0000","last_active":"2024-04-25T01:43:22.431720+0000","last_peered":"2024-04-25T01:43:22.431720+0000","last_clean":"2024-04-25T01:43:22.431720+0000","last_became_active":"2024-04-25T01:43:17.424364+0000","last_became_peered":"2024-04-25T01:43:17.424364+0000","last_unstale":"2024-04-25T01:43:22.431720+0000","last_undegraded":"2024-04-25T01:43:22.431720+0000","last_fullsized":"2024-04-25T01:43:22.431720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:16.699513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:43.422151+0000","last_change":"2024-04-25T01:43:17.028904+0000","last_active":"2024-04-25T01:43:43.422151+0000","last_peered":"2024-04-25T01:43:43.422151+0000","last_clean":"2024-04-25T01:43:43.422151+0000","last_became_active":"2024-04-25T01:43:17.028608+0000","last_became_peered":"2024-04-25T01:43:17.028608+0000","last_unstale":"2024-04-25T01:43:43.422151+0000","last_undegraded":"2024-04-25T01:43:43.422151+0000","last_fullsized":"2024-04-25T01:43:43.422151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:19.297926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157882+0000","last_change":"2024-04-25T01:43:21.440009+0000","last_active":"2024-04-25T01:43:23.157882+0000","last_peered":"2024-04-25T01:43:23.157882+0000","last_clean":"2024-04-25T01:43:23.157882+0000","last_became_active":"2024-04-25T01:43:21.439611+0000","last_became_peered":"2024-04-25T01:43:21.439611+0000","last_unstale":"2024-04-25T01:43:23.157882+0000","last_undegraded":"2024-04-25T01:43:23.157882+0000","last_fullsized":"2024-04-25T01:43:23.157882+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:14.952044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112678+0000","last_change":"2024-04-25T01:43:17.428636+0000","last_active":"2024-04-25T01:43:23.112678+0000","last_peered":"2024-04-25T01:43:23.112678+0000","last_clean":"2024-04-25T01:43:23.112678+0000","last_became_active":"2024-04-25T01:43:17.428316+0000","last_became_peered":"2024-04-25T01:43:17.428316+0000","last_unstale":"2024-04-25T01:43:23.112678+0000","last_undegraded":"2024-04-25T01:43:23.112678+0000","last_fullsized":"2024-04-25T01:43:23.112678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:04.683846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.424045+0000","last_change":"2024-04-25T01:43:17.020632+0000","last_active":"2024-04-25T01:43:42.424045+0000","last_peered":"2024-04-25T01:43:42.424045+0000","last_clean":"2024-04-25T01:43:42.424045+0000","last_became_active":"2024-04-25T01:43:17.020278+0000","last_became_peered":"2024-04-25T01:43:17.020278+0000","last_unstale":"2024-04-25T01:43:42.424045+0000","last_undegraded":"2024-04-25T01:43:42.424045+0000","last_fullsized":"2024-04-25T01:43:42.424045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:13.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":347,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157176+0000","last_change":"2024-04-25T01:42:05.003964+0000","last_active":"2024-04-25T01:43:23.157176+0000","last_peered":"2024-04-25T01:43:23.157176+0000","last_clean":"2024-04-25T01:43:23.157176+0000","last_became_active":"2024-04-25T01:42:05.003509+0000","last_became_peered":"2024-04-25T01:42:05.003509+0000","last_unstale":"2024-04-25T01:43:23.157176+0000","last_undegraded":"2024-04-25T01:43:23.157176+0000","last_fullsized":"2024-04-25T01:43:23.157176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:38:39.137002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:38:39.137002+0000","last_clean_scrub_stamp":"2024-04-25T01:38:39.137002+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:53.878838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.432168+0000","last_change":"2024-04-25T01:43:21.626345+0000","last_active":"2024-04-25T01:43:22.432168+0000","last_peered":"2024-04-25T01:43:22.432168+0000","last_clean":"2024-04-25T01:43:22.432168+0000","last_became_active":"2024-04-25T01:43:21.625884+0000","last_became_peered":"2024-04-25T01:43:21.625884+0000","last_unstale":"2024-04-25T01:43:22.432168+0000","last_undegraded":"2024-04-25T01:43:22.432168+0000","last_fullsized":"2024-04-25T01:43:22.432168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:37.961617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.430581+0000","last_change":"2024-04-25T01:43:17.421186+0000","last_active":"2024-04-25T01:43:22.430581+0000","last_peered":"2024-04-25T01:43:22.430581+0000","last_clean":"2024-04-25T01:43:22.430581+0000","last_became_active":"2024-04-25T01:43:17.420824+0000","last_became_peered":"2024-04-25T01:43:17.420824+0000","last_unstale":"2024-04-25T01:43:22.430581+0000","last_undegraded":"2024-04-25T01:43:22.430581+0000","last_fullsized":"2024-04-25T01:43:22.430581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:27.829126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.006961+0000","last_change":"2024-04-25T01:43:17.034050+0000","last_active":"2024-04-25T01:43:42.006961+0000","last_peered":"2024-04-25T01:43:42.006961+0000","last_clean":"2024-04-25T01:43:42.006961+0000","last_became_active":"2024-04-25T01:43:17.033725+0000","last_became_peered":"2024-04-25T01:43:17.033725+0000","last_unstale":"2024-04-25T01:43:42.006961+0000","last_undegraded":"2024-04-25T01:43:42.006961+0000","last_fullsized":"2024-04-25T01:43:42.006961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:06.419265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157725+0000","last_change":"2024-04-25T01:43:21.443299+0000","last_active":"2024-04-25T01:43:23.157725+0000","last_peered":"2024-04-25T01:43:23.157725+0000","last_clean":"2024-04-25T01:43:23.157725+0000","last_became_active":"2024-04-25T01:43:21.442899+0000","last_became_peered":"2024-04-25T01:43:21.442899+0000","last_unstale":"2024-04-25T01:43:23.157725+0000","last_undegraded":"2024-04-25T01:43:23.157725+0000","last_fullsized":"2024-04-25T01:43:23.157725+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:51.720208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721602+0000","last_change":"2024-04-25T01:43:17.035136+0000","last_active":"2024-04-25T01:43:22.721602+0000","last_peered":"2024-04-25T01:43:22.721602+0000","last_clean":"2024-04-25T01:43:22.721602+0000","last_became_active":"2024-04-25T01:43:17.034289+0000","last_became_peered":"2024-04-25T01:43:17.034289+0000","last_unstale":"2024-04-25T01:43:22.721602+0000","last_undegraded":"2024-04-25T01:43:22.721602+0000","last_fullsized":"2024-04-25T01:43:22.721602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:07.450025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431092+0000","last_change":"2024-04-25T01:43:17.414478+0000","last_active":"2024-04-25T01:43:22.431092+0000","last_peered":"2024-04-25T01:43:22.431092+0000","last_clean":"2024-04-25T01:43:22.431092+0000","last_became_active":"2024-04-25T01:43:17.414061+0000","last_became_peered":"2024-04-25T01:43:17.414061+0000","last_unstale":"2024-04-25T01:43:22.431092+0000","last_undegraded":"2024-04-25T01:43:22.431092+0000","last_fullsized":"2024-04-25T01:43:22.431092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:16.051518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112618+0000","last_change":"2024-04-25T01:43:21.625963+0000","last_active":"2024-04-25T01:43:23.112618+0000","last_peered":"2024-04-25T01:43:23.112618+0000","last_clean":"2024-04-25T01:43:23.112618+0000","last_became_active":"2024-04-25T01:43:21.625563+0000","last_became_peered":"2024-04-25T01:43:21.625563+0000","last_unstale":"2024-04-25T01:43:23.112618+0000","last_undegraded":"2024-04-25T01:43:23.112618+0000","last_fullsized":"2024-04-25T01:43:23.112618+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:11.217504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157738+0000","last_change":"2024-04-25T01:43:17.024391+0000","last_active":"2024-04-25T01:43:23.157738+0000","last_peered":"2024-04-25T01:43:23.157738+0000","last_clean":"2024-04-25T01:43:23.157738+0000","last_became_active":"2024-04-25T01:43:17.023994+0000","last_became_peered":"2024-04-25T01:43:17.023994+0000","last_unstale":"2024-04-25T01:43:23.157738+0000","last_undegraded":"2024-04-25T01:43:23.157738+0000","last_fullsized":"2024-04-25T01:43:23.157738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:56.489383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.113025+0000","last_change":"2024-04-25T01:43:17.435357+0000","last_active":"2024-04-25T01:43:23.113025+0000","last_peered":"2024-04-25T01:43:23.113025+0000","last_clean":"2024-04-25T01:43:23.113025+0000","last_became_active":"2024-04-25T01:43:17.435090+0000","last_became_peered":"2024-04-25T01:43:17.435090+0000","last_unstale":"2024-04-25T01:43:23.113025+0000","last_undegraded":"2024-04-25T01:43:23.113025+0000","last_fullsized":"2024-04-25T01:43:23.113025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:49.181850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158097+0000","last_change":"2024-04-25T01:43:21.441314+0000","last_active":"2024-04-25T01:43:23.158097+0000","last_peered":"2024-04-25T01:43:23.158097+0000","last_clean":"2024-04-25T01:43:23.158097+0000","last_became_active":"2024-04-25T01:43:21.440896+0000","last_became_peered":"2024-04-25T01:43:21.440896+0000","last_unstale":"2024-04-25T01:43:23.158097+0000","last_undegraded":"2024-04-25T01:43:23.158097+0000","last_fullsized":"2024-04-25T01:43:23.158097+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:36.091128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:43.440878+0000","last_change":"2024-04-25T01:43:17.031525+0000","last_active":"2024-04-25T01:43:43.440878+0000","last_peered":"2024-04-25T01:43:43.440878+0000","last_clean":"2024-04-25T01:43:43.440878+0000","last_became_active":"2024-04-25T01:43:17.031203+0000","last_became_peered":"2024-04-25T01:43:17.031203+0000","last_unstale":"2024-04-25T01:43:43.440878+0000","last_undegraded":"2024-04-25T01:43:43.440878+0000","last_fullsized":"2024-04-25T01:43:43.440878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:22.929179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157329+0000","last_change":"2024-04-25T01:43:17.425981+0000","last_active":"2024-04-25T01:43:23.157329+0000","last_peered":"2024-04-25T01:43:23.157329+0000","last_clean":"2024-04-25T01:43:23.157329+0000","last_became_active":"2024-04-25T01:43:17.425674+0000","last_became_peered":"2024-04-25T01:43:17.425674+0000","last_unstale":"2024-04-25T01:43:23.157329+0000","last_undegraded":"2024-04-25T01:43:23.157329+0000","last_fullsized":"2024-04-25T01:43:23.157329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:58.441774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158326+0000","last_change":"2024-04-25T01:43:21.626389+0000","last_active":"2024-04-25T01:43:23.158326+0000","last_peered":"2024-04-25T01:43:23.158326+0000","last_clean":"2024-04-25T01:43:23.158326+0000","last_became_active":"2024-04-25T01:43:21.625859+0000","last_became_peered":"2024-04-25T01:43:21.625859+0000","last_unstale":"2024-04-25T01:43:23.158326+0000","last_undegraded":"2024-04-25T01:43:23.158326+0000","last_fullsized":"2024-04-25T01:43:23.158326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:51.372126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.429851+0000","last_change":"2024-04-25T01:43:17.022903+0000","last_active":"2024-04-25T01:43:22.429851+0000","last_peered":"2024-04-25T01:43:22.429851+0000","last_clean":"2024-04-25T01:43:22.429851+0000","last_became_active":"2024-04-25T01:43:17.022564+0000","last_became_peered":"2024-04-25T01:43:17.022564+0000","last_unstale":"2024-04-25T01:43:22.429851+0000","last_undegraded":"2024-04-25T01:43:22.429851+0000","last_fullsized":"2024-04-25T01:43:22.429851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:01.277500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721183+0000","last_change":"2024-04-25T01:43:17.422855+0000","last_active":"2024-04-25T01:43:22.721183+0000","last_peered":"2024-04-25T01:43:22.721183+0000","last_clean":"2024-04-25T01:43:22.721183+0000","last_became_active":"2024-04-25T01:43:17.422490+0000","last_became_peered":"2024-04-25T01:43:17.422490+0000","last_unstale":"2024-04-25T01:43:22.721183+0000","last_undegraded":"2024-04-25T01:43:22.721183+0000","last_fullsized":"2024-04-25T01:43:22.721183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:40.016044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157958+0000","last_change":"2024-04-25T01:43:21.440080+0000","last_active":"2024-04-25T01:43:23.157958+0000","last_peered":"2024-04-25T01:43:23.157958+0000","last_clean":"2024-04-25T01:43:23.157958+0000","last_became_active":"2024-04-25T01:43:21.439669+0000","last_became_peered":"2024-04-25T01:43:21.439669+0000","last_unstale":"2024-04-25T01:43:23.157958+0000","last_undegraded":"2024-04-25T01:43:23.157958+0000","last_fullsized":"2024-04-25T01:43:23.157958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:34.143210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.423564+0000","last_change":"2024-04-25T01:43:17.035521+0000","last_active":"2024-04-25T01:43:42.423564+0000","last_peered":"2024-04-25T01:43:42.423564+0000","last_clean":"2024-04-25T01:43:42.423564+0000","last_became_active":"2024-04-25T01:43:17.033762+0000","last_became_peered":"2024-04-25T01:43:17.033762+0000","last_unstale":"2024-04-25T01:43:42.423564+0000","last_undegraded":"2024-04-25T01:43:42.423564+0000","last_fullsized":"2024-04-25T01:43:42.423564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:04.762578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157415+0000","last_change":"2024-04-25T01:43:17.426952+0000","last_active":"2024-04-25T01:43:23.157415+0000","last_peered":"2024-04-25T01:43:23.157415+0000","last_clean":"2024-04-25T01:43:23.157415+0000","last_became_active":"2024-04-25T01:43:17.426573+0000","last_became_peered":"2024-04-25T01:43:17.426573+0000","last_unstale":"2024-04-25T01:43:23.157415+0000","last_undegraded":"2024-04-25T01:43:23.157415+0000","last_fullsized":"2024-04-25T01:43:23.157415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:33.807254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158247+0000","last_change":"2024-04-25T01:43:21.453295+0000","last_active":"2024-04-25T01:43:23.158247+0000","last_peered":"2024-04-25T01:43:23.158247+0000","last_clean":"2024-04-25T01:43:23.158247+0000","last_became_active":"2024-04-25T01:43:21.452031+0000","last_became_peered":"2024-04-25T01:43:21.452031+0000","last_unstale":"2024-04-25T01:43:23.158247+0000","last_undegraded":"2024-04-25T01:43:23.158247+0000","last_fullsized":"2024-04-25T01:43:23.158247+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:23.956638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:41.995947+0000","last_change":"2024-04-25T01:43:17.029253+0000","last_active":"2024-04-25T01:43:41.995947+0000","last_peered":"2024-04-25T01:43:41.995947+0000","last_clean":"2024-04-25T01:43:41.995947+0000","last_became_active":"2024-04-25T01:43:17.028949+0000","last_became_peered":"2024-04-25T01:43:17.028949+0000","last_unstale":"2024-04-25T01:43:41.995947+0000","last_undegraded":"2024-04-25T01:43:41.995947+0000","last_fullsized":"2024-04-25T01:43:41.995947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:05.101916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112744+0000","last_change":"2024-04-25T01:43:17.431319+0000","last_active":"2024-04-25T01:43:23.112744+0000","last_peered":"2024-04-25T01:43:23.112744+0000","last_clean":"2024-04-25T01:43:23.112744+0000","last_became_active":"2024-04-25T01:43:17.431007+0000","last_became_peered":"2024-04-25T01:43:17.431007+0000","last_unstale":"2024-04-25T01:43:23.112744+0000","last_undegraded":"2024-04-25T01:43:23.112744+0000","last_fullsized":"2024-04-25T01:43:23.112744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:38.999286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158416+0000","last_change":"2024-04-25T01:43:21.625931+0000","last_active":"2024-04-25T01:43:23.158416+0000","last_peered":"2024-04-25T01:43:23.158416+0000","last_clean":"2024-04-25T01:43:23.158416+0000","last_became_active":"2024-04-25T01:43:21.625460+0000","last_became_peered":"2024-04-25T01:43:21.625460+0000","last_unstale":"2024-04-25T01:43:23.158416+0000","last_undegraded":"2024-04-25T01:43:23.158416+0000","last_fullsized":"2024-04-25T01:43:23.158416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:59.665909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721268+0000","last_change":"2024-04-25T01:43:17.033937+0000","last_active":"2024-04-25T01:43:22.721268+0000","last_peered":"2024-04-25T01:43:22.721268+0000","last_clean":"2024-04-25T01:43:22.721268+0000","last_became_active":"2024-04-25T01:43:17.033629+0000","last_became_peered":"2024-04-25T01:43:17.033629+0000","last_unstale":"2024-04-25T01:43:22.721268+0000","last_undegraded":"2024-04-25T01:43:22.721268+0000","last_fullsized":"2024-04-25T01:43:22.721268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:05.531624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157856+0000","last_change":"2024-04-25T01:43:17.436068+0000","last_active":"2024-04-25T01:43:23.157856+0000","last_peered":"2024-04-25T01:43:23.157856+0000","last_clean":"2024-04-25T01:43:23.157856+0000","last_became_active":"2024-04-25T01:43:17.435383+0000","last_became_peered":"2024-04-25T01:43:17.435383+0000","last_unstale":"2024-04-25T01:43:23.157856+0000","last_undegraded":"2024-04-25T01:43:23.157856+0000","last_fullsized":"2024-04-25T01:43:23.157856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:00.087923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.429770+0000","last_change":"2024-04-25T01:43:21.441696+0000","last_active":"2024-04-25T01:43:22.429770+0000","last_peered":"2024-04-25T01:43:22.429770+0000","last_clean":"2024-04-25T01:43:22.429770+0000","last_became_active":"2024-04-25T01:43:21.441263+0000","last_became_peered":"2024-04-25T01:43:21.441263+0000","last_unstale":"2024-04-25T01:43:22.429770+0000","last_undegraded":"2024-04-25T01:43:22.429770+0000","last_fullsized":"2024-04-25T01:43:22.429770+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:57.263465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.427281+0000","last_change":"2024-04-25T01:43:17.033841+0000","last_active":"2024-04-25T01:43:42.427281+0000","last_peered":"2024-04-25T01:43:42.427281+0000","last_clean":"2024-04-25T01:43:42.427281+0000","last_became_active":"2024-04-25T01:43:17.033477+0000","last_became_peered":"2024-04-25T01:43:17.033477+0000","last_unstale":"2024-04-25T01:43:42.427281+0000","last_undegraded":"2024-04-25T01:43:42.427281+0000","last_fullsized":"2024-04-25T01:43:42.427281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:43.717525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158025+0000","last_change":"2024-04-25T01:43:17.430744+0000","last_active":"2024-04-25T01:43:23.158025+0000","last_peered":"2024-04-25T01:43:23.158025+0000","last_clean":"2024-04-25T01:43:23.158025+0000","last_became_active":"2024-04-25T01:43:17.430434+0000","last_became_peered":"2024-04-25T01:43:17.430434+0000","last_unstale":"2024-04-25T01:43:23.158025+0000","last_undegraded":"2024-04-25T01:43:23.158025+0000","last_fullsized":"2024-04-25T01:43:23.158025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:17.895901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.156878+0000","last_change":"2024-04-25T01:43:21.451604+0000","last_active":"2024-04-25T01:43:23.156878+0000","last_peered":"2024-04-25T01:43:23.156878+0000","last_clean":"2024-04-25T01:43:23.156878+0000","last_became_active":"2024-04-25T01:43:21.451178+0000","last_became_peered":"2024-04-25T01:43:21.451178+0000","last_unstale":"2024-04-25T01:43:23.156878+0000","last_undegraded":"2024-04-25T01:43:23.156878+0000","last_fullsized":"2024-04-25T01:43:23.156878+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:17.387498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.423954+0000","last_change":"2024-04-25T01:43:17.025580+0000","last_active":"2024-04-25T01:43:42.423954+0000","last_peered":"2024-04-25T01:43:42.423954+0000","last_clean":"2024-04-25T01:43:42.423954+0000","last_became_active":"2024-04-25T01:43:17.025267+0000","last_became_peered":"2024-04-25T01:43:17.025267+0000","last_unstale":"2024-04-25T01:43:42.423954+0000","last_undegraded":"2024-04-25T01:43:42.423954+0000","last_fullsized":"2024-04-25T01:43:42.423954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:04.922572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157533+0000","last_change":"2024-04-25T01:43:17.427502+0000","last_active":"2024-04-25T01:43:23.157533+0000","last_peered":"2024-04-25T01:43:23.157533+0000","last_clean":"2024-04-25T01:43:23.157533+0000","last_became_active":"2024-04-25T01:43:17.427149+0000","last_became_peered":"2024-04-25T01:43:17.427149+0000","last_unstale":"2024-04-25T01:43:23.157533+0000","last_undegraded":"2024-04-25T01:43:23.157533+0000","last_fullsized":"2024-04-25T01:43:23.157533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:49.292709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431300+0000","last_change":"2024-04-25T01:43:21.437988+0000","last_active":"2024-04-25T01:43:22.431300+0000","last_peered":"2024-04-25T01:43:22.431300+0000","last_clean":"2024-04-25T01:43:22.431300+0000","last_became_active":"2024-04-25T01:43:21.437523+0000","last_became_peered":"2024-04-25T01:43:21.437523+0000","last_unstale":"2024-04-25T01:43:22.431300+0000","last_undegraded":"2024-04-25T01:43:22.431300+0000","last_fullsized":"2024-04-25T01:43:22.431300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:09.076123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:43.442351+0000","last_change":"2024-04-25T01:43:17.023058+0000","last_active":"2024-04-25T01:43:43.442351+0000","last_peered":"2024-04-25T01:43:43.442351+0000","last_clean":"2024-04-25T01:43:43.442351+0000","last_became_active":"2024-04-25T01:43:17.022730+0000","last_became_peered":"2024-04-25T01:43:17.022730+0000","last_unstale":"2024-04-25T01:43:43.442351+0000","last_undegraded":"2024-04-25T01:43:43.442351+0000","last_fullsized":"2024-04-25T01:43:43.442351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:12.279161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158616+0000","last_change":"2024-04-25T01:43:17.428832+0000","last_active":"2024-04-25T01:43:23.158616+0000","last_peered":"2024-04-25T01:43:23.158616+0000","last_clean":"2024-04-25T01:43:23.158616+0000","last_became_active":"2024-04-25T01:43:17.428493+0000","last_became_peered":"2024-04-25T01:43:17.428493+0000","last_unstale":"2024-04-25T01:43:23.158616+0000","last_undegraded":"2024-04-25T01:43:23.158616+0000","last_fullsized":"2024-04-25T01:43:23.158616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:47.224987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157502+0000","last_change":"2024-04-25T01:43:17.429227+0000","last_active":"2024-04-25T01:43:23.157502+0000","last_peered":"2024-04-25T01:43:23.157502+0000","last_clean":"2024-04-25T01:43:23.157502+0000","last_became_active":"2024-04-25T01:43:17.428693+0000","last_became_peered":"2024-04-25T01:43:17.428693+0000","last_unstale":"2024-04-25T01:43:23.157502+0000","last_undegraded":"2024-04-25T01:43:23.157502+0000","last_fullsized":"2024-04-25T01:43:23.157502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:58.511414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157309+0000","last_change":"2024-04-25T01:43:17.425880+0000","last_active":"2024-04-25T01:43:23.157309+0000","last_peered":"2024-04-25T01:43:23.157309+0000","last_clean":"2024-04-25T01:43:23.157309+0000","last_became_active":"2024-04-25T01:43:17.425539+0000","last_became_peered":"2024-04-25T01:43:17.425539+0000","last_unstale":"2024-04-25T01:43:23.157309+0000","last_undegraded":"2024-04-25T01:43:23.157309+0000","last_fullsized":"2024-04-25T01:43:23.157309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:40.909991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157272+0000","last_change":"2024-04-25T01:43:17.421214+0000","last_active":"2024-04-25T01:43:23.157272+0000","last_peered":"2024-04-25T01:43:23.157272+0000","last_clean":"2024-04-25T01:43:23.157272+0000","last_became_active":"2024-04-25T01:43:17.420872+0000","last_became_peered":"2024-04-25T01:43:17.420872+0000","last_unstale":"2024-04-25T01:43:23.157272+0000","last_undegraded":"2024-04-25T01:43:23.157272+0000","last_fullsized":"2024-04-25T01:43:23.157272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:48.066191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157056+0000","last_change":"2024-04-25T01:43:17.426011+0000","last_active":"2024-04-25T01:43:23.157056+0000","last_peered":"2024-04-25T01:43:23.157056+0000","last_clean":"2024-04-25T01:43:23.157056+0000","last_became_active":"2024-04-25T01:43:17.425629+0000","last_became_peered":"2024-04-25T01:43:17.425629+0000","last_unstale":"2024-04-25T01:43:23.157056+0000","last_undegraded":"2024-04-25T01:43:23.157056+0000","last_fullsized":"2024-04-25T01:43:23.157056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:12.606749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431170+0000","last_change":"2024-04-25T01:43:17.421374+0000","last_active":"2024-04-25T01:43:22.431170+0000","last_peered":"2024-04-25T01:43:22.431170+0000","last_clean":"2024-04-25T01:43:22.431170+0000","last_became_active":"2024-04-25T01:43:17.420864+0000","last_became_peered":"2024-04-25T01:43:17.420864+0000","last_unstale":"2024-04-25T01:43:22.431170+0000","last_undegraded":"2024-04-25T01:43:22.431170+0000","last_fullsized":"2024-04-25T01:43:22.431170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:17.517927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157716+0000","last_change":"2024-04-25T01:43:17.431131+0000","last_active":"2024-04-25T01:43:23.157716+0000","last_peered":"2024-04-25T01:43:23.157716+0000","last_clean":"2024-04-25T01:43:23.157716+0000","last_became_active":"2024-04-25T01:43:17.430809+0000","last_became_peered":"2024-04-25T01:43:17.430809+0000","last_unstale":"2024-04-25T01:43:23.157716+0000","last_undegraded":"2024-04-25T01:43:23.157716+0000","last_fullsized":"2024-04-25T01:43:23.157716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:48.261501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158423+0000","last_change":"2024-04-25T01:43:17.419104+0000","last_active":"2024-04-25T01:43:23.158423+0000","last_peered":"2024-04-25T01:43:23.158423+0000","last_clean":"2024-04-25T01:43:23.158423+0000","last_became_active":"2024-04-25T01:43:17.418799+0000","last_became_peered":"2024-04-25T01:43:17.418799+0000","last_unstale":"2024-04-25T01:43:23.158423+0000","last_undegraded":"2024-04-25T01:43:23.158423+0000","last_fullsized":"2024-04-25T01:43:23.158423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:46.000847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.113092+0000","last_change":"2024-04-25T01:43:17.425801+0000","last_active":"2024-04-25T01:43:23.113092+0000","last_peered":"2024-04-25T01:43:23.113092+0000","last_clean":"2024-04-25T01:43:23.113092+0000","last_became_active":"2024-04-25T01:43:17.425133+0000","last_became_peered":"2024-04-25T01:43:17.425133+0000","last_unstale":"2024-04-25T01:43:23.113092+0000","last_undegraded":"2024-04-25T01:43:23.113092+0000","last_fullsized":"2024-04-25T01:43:23.113092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:54.229265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431493+0000","last_change":"2024-04-25T01:43:17.423931+0000","last_active":"2024-04-25T01:43:22.431493+0000","last_peered":"2024-04-25T01:43:22.431493+0000","last_clean":"2024-04-25T01:43:22.431493+0000","last_became_active":"2024-04-25T01:43:17.423477+0000","last_became_peered":"2024-04-25T01:43:17.423477+0000","last_unstale":"2024-04-25T01:43:22.431493+0000","last_undegraded":"2024-04-25T01:43:22.431493+0000","last_fullsized":"2024-04-25T01:43:22.431493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:31.538936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112801+0000","last_change":"2024-04-25T01:43:17.428469+0000","last_active":"2024-04-25T01:43:23.112801+0000","last_peered":"2024-04-25T01:43:23.112801+0000","last_clean":"2024-04-25T01:43:23.112801+0000","last_became_active":"2024-04-25T01:43:17.428121+0000","last_became_peered":"2024-04-25T01:43:17.428121+0000","last_unstale":"2024-04-25T01:43:23.112801+0000","last_undegraded":"2024-04-25T01:43:23.112801+0000","last_fullsized":"2024-04-25T01:43:23.112801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:27.563439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.430016+0000","last_change":"2024-04-25T01:43:17.433014+0000","last_active":"2024-04-25T01:43:22.430016+0000","last_peered":"2024-04-25T01:43:22.430016+0000","last_clean":"2024-04-25T01:43:22.430016+0000","last_became_active":"2024-04-25T01:43:17.432568+0000","last_became_peered":"2024-04-25T01:43:17.432568+0000","last_unstale":"2024-04-25T01:43:22.430016+0000","last_undegraded":"2024-04-25T01:43:22.430016+0000","last_fullsized":"2024-04-25T01:43:22.430016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:18.494922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.430921+0000","last_change":"2024-04-25T01:43:17.436079+0000","last_active":"2024-04-25T01:43:22.430921+0000","last_peered":"2024-04-25T01:43:22.430921+0000","last_clean":"2024-04-25T01:43:22.430921+0000","last_became_active":"2024-04-25T01:43:17.435735+0000","last_became_peered":"2024-04-25T01:43:17.435735+0000","last_unstale":"2024-04-25T01:43:22.430921+0000","last_undegraded":"2024-04-25T01:43:22.430921+0000","last_fullsized":"2024-04-25T01:43:22.430921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:59.593710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157724+0000","last_change":"2024-04-25T01:43:17.424956+0000","last_active":"2024-04-25T01:43:23.157724+0000","last_peered":"2024-04-25T01:43:23.157724+0000","last_clean":"2024-04-25T01:43:23.157724+0000","last_became_active":"2024-04-25T01:43:17.424629+0000","last_became_peered":"2024-04-25T01:43:17.424629+0000","last_unstale":"2024-04-25T01:43:23.157724+0000","last_undegraded":"2024-04-25T01:43:23.157724+0000","last_fullsized":"2024-04-25T01:43:23.157724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:58.006952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245924,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9863,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":131,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":45883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":482901,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612631,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1076875,"data_compressed":50529,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776156,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":482291,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":482313,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1074932,"data_compressed":50529,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462956,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":482431,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":485551,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":481799,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":487523,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":481993,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":482800,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724593,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":1075288,"data_compressed":50529,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:43:45.791 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:43:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.931+0000 7f35f86cd640 1 -- 172.21.15.78:0/2001251061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f0102230 msgr2=0x7f35f0108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.931+0000 7f35f86cd640 1 --2- 172.21.15.78:0/2001251061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f0102230 0x7f35f0108ac0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009ca0 tx=0x7f35e802a8c0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 -- 172.21.15.78:0/2001251061 shutdown_connections 2024-04-25T01:43:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 --2- 172.21.15.78:0/2001251061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f0102230 0x7f35f0108ac0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 --2- 172.21.15.78:0/2001251061 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101870 0x7f35f0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 --2- 172.21.15.78:0/2001251061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f0100680 0x7f35f0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 -- 172.21.15.78:0/2001251061 >> 172.21.15.78:0/2001251061 conn(0x7f35f00fbbf0 msgr2=0x7f35f00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 -- 172.21.15.78:0/2001251061 shutdown_connections 2024-04-25T01:43:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 -- 172.21.15.78:0/2001251061 wait complete. 2024-04-25T01:43:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 Processor -- start 2024-04-25T01:43:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 -- start start 2024-04-25T01:43:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f0100680 0x7f35f0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101870 0x7f35f0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f0102230 0x7f35f019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35f010c370 con 0x7f35f0102230 2024-04-25T01:43:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f35f010c1f0 con 0x7f35f0101870 2024-04-25T01:43:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f86cd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35f010c4f0 con 0x7f35f0100680 2024-04-25T01:43:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f6c6a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f0102230 0x7f35f019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f6c6a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f0102230 0x7f35f019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58646/0 (socket says 172.21.15.78:58646) 2024-04-25T01:43:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f6c6a640 1 -- 172.21.15.78:0/443294661 learned_addr learned my addr 172.21.15.78:0/443294661 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f5c68640 1 --2- 172.21.15.78:0/443294661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101870 0x7f35f0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.932+0000 7f35f6469640 1 --2- 172.21.15.78:0/443294661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f0100680 0x7f35f0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.933+0000 7f35f6c6a640 1 -- 172.21.15.78:0/443294661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f0100680 msgr2=0x7f35f0197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.933+0000 7f35f6c6a640 1 --2- 172.21.15.78:0/443294661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f0100680 0x7f35f0197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.933+0000 7f35f6c6a640 1 -- 172.21.15.78:0/443294661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101870 msgr2=0x7f35f0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.933+0000 7f35f6c6a640 1 --2- 172.21.15.78:0/443294661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101870 0x7f35f0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.933+0000 7f35f6c6a640 1 -- 172.21.15.78:0/443294661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f019f760 con 0x7f35f0102230 2024-04-25T01:43:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.933+0000 7f35f6c6a640 1 --2- 172.21.15.78:0/443294661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f0102230 0x7f35f019f0f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f35e8005e30 tx=0x7f35e8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.933+0000 7f35e77fe640 1 -- 172.21.15.78:0/443294661 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8003b70 con 0x7f35f0102230 2024-04-25T01:43:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.933+0000 7f35f86cd640 1 -- 172.21.15.78:0/443294661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f019f9f0 con 0x7f35f0102230 2024-04-25T01:43:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.933+0000 7f35e77fe640 1 -- 172.21.15.78:0/443294661 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e8003d10 con 0x7f35f0102230 2024-04-25T01:43:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.933+0000 7f35e77fe640 1 -- 172.21.15.78:0/443294661 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e802fd30 con 0x7f35f0102230 2024-04-25T01:43:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.933+0000 7f35f86cd640 1 -- 172.21.15.78:0/443294661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f019ff40 con 0x7f35f0102230 2024-04-25T01:43:45.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.934+0000 7f35f86cd640 1 -- 172.21.15.78:0/443294661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f01a0170 con 0x7f35f0102230 2024-04-25T01:43:45.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.935+0000 7f35e77fe640 1 -- 172.21.15.78:0/443294661 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f35e8004050 con 0x7f35f0102230 2024-04-25T01:43:45.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.935+0000 7f35e77fe640 1 --2- 172.21.15.78:0/443294661 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f35dc0a2c20 0x7f35dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:45.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.936+0000 7f35f6469640 1 --2- 172.21.15.78:0/443294661 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f35dc0a2c20 0x7f35dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:45.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.936+0000 7f35e77fe640 1 -- 172.21.15.78:0/443294661 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f35e80f0500 con 0x7f35f0102230 2024-04-25T01:43:45.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.936+0000 7f35f6469640 1 --2- 172.21.15.78:0/443294661 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f35dc0a2c20 0x7f35dc0a50e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f35f01016d0 tx=0x7f35e0009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:45.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:45.940+0000 7f35e77fe640 1 -- 172.21.15.78:0/443294661 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f35e80bd2f0 con 0x7f35f0102230 2024-04-25T01:43:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:46.112+0000 7f35f86cd640 1 -- 172.21.15.78:0/443294661 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35f01a0340 con 0x7f35dc0a2c20 2024-04-25T01:43:46.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:46.121+0000 7f35e77fe640 1 -- 172.21.15.78:0/443294661 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340583 (secure 0 0 0) 0x7f35f01a0340 con 0x7f35dc0a2c20 2024-04-25T01:43:46.116 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-04-25T01:43:46.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:46.125+0000 7f35f86cd640 1 -- 172.21.15.78:0/443294661 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f35dc0a2c20 msgr2=0x7f35dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:46.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:46.125+0000 7f35f86cd640 1 --2- 172.21.15.78:0/443294661 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f35dc0a2c20 0x7f35dc0a50e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f35f01016d0 tx=0x7f35e0009110 comp rx=0 tx=0).stop 2024-04-25T01:43:46.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:46.125+0000 7f35f86cd640 1 -- 172.21.15.78:0/443294661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f0102230 msgr2=0x7f35f019f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:46.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:46.125+0000 7f35f86cd640 1 --2- 172.21.15.78:0/443294661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f0102230 0x7f35f019f0f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f35e8005e30 tx=0x7f35e8031ee0 comp rx=0 tx=0).stop 2024-04-25T01:43:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:46.125+0000 7f35f86cd640 1 -- 172.21.15.78:0/443294661 shutdown_connections 2024-04-25T01:43:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:46.125+0000 7f35f86cd640 1 --2- 172.21.15.78:0/443294661 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f35dc0a2c20 0x7f35dc0a50e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:46.125+0000 7f35f86cd640 1 --2- 172.21.15.78:0/443294661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f0102230 0x7f35f019f0f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:46.125+0000 7f35f86cd640 1 --2- 172.21.15.78:0/443294661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101870 0x7f35f0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:46.125+0000 7f35f86cd640 1 --2- 172.21.15.78:0/443294661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f0100680 0x7f35f0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:46.125+0000 7f35f86cd640 1 -- 172.21.15.78:0/443294661 >> 172.21.15.78:0/443294661 conn(0x7f35f00fbbf0 msgr2=0x7f35f00688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:46.125+0000 7f35f86cd640 1 -- 172.21.15.78:0/443294661 shutdown_connections 2024-04-25T01:43:46.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:46.125+0000 7f35f86cd640 1 -- 172.21.15.78:0/443294661 wait complete. 2024-04-25T01:43:46.126 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:45 smithi078 ceph-mon[29469]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:46.126 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:45 smithi078 ceph-mon[29469]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:46.126 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:45 smithi078 ceph-mon[29469]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:46.126 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:45 smithi078 ceph-mon[29469]: pgmap v304: 129 pgs: 129 active+clean; 586 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-25T01:43:46.126 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:45 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/33290992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:46.126 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:45 smithi078 ceph-mon[29469]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T01:43:46.126 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:45 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/489610415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:46.127 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-04-25T01:43:44.899041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600231,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":145,"num_read_kb":124,"num_write":312,"num_write_kb":2993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":256,"ondisk_log_size":256,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360980,"kb_used_data":11808,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580844,"statfs":{"total":1151940427776,"available":1151570784256,"internally_reserved":0,"allocated":12091392,"data_stored":7576697,"data_compressed":283734,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5364,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":84,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003652"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158208+0000","last_change":"2024-04-25T01:43:17.423128+0000","last_active":"2024-04-25T01:43:23.158208+0000","last_peered":"2024-04-25T01:43:23.158208+0000","last_clean":"2024-04-25T01:43:23.158208+0000","last_became_active":"2024-04-25T01:43:17.422356+0000","last_became_peered":"2024-04-25T01:43:17.422356+0000","last_unstale":"2024-04-25T01:43:23.158208+0000","last_undegraded":"2024-04-25T01:43:23.158208+0000","last_fullsized":"2024-04-25T01:43:23.158208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:05.524727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157427+0000","last_change":"2024-04-25T01:43:17.422866+0000","last_active":"2024-04-25T01:43:23.157427+0000","last_peered":"2024-04-25T01:43:23.157427+0000","last_clean":"2024-04-25T01:43:23.157427+0000","last_became_active":"2024-04-25T01:43:17.422574+0000","last_became_peered":"2024-04-25T01:43:17.422574+0000","last_unstale":"2024-04-25T01:43:23.157427+0000","last_undegraded":"2024-04-25T01:43:23.157427+0000","last_fullsized":"2024-04-25T01:43:23.157427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.842361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157305+0000","last_change":"2024-04-25T01:43:17.419320+0000","last_active":"2024-04-25T01:43:23.157305+0000","last_peered":"2024-04-25T01:43:23.157305+0000","last_clean":"2024-04-25T01:43:23.157305+0000","last_became_active":"2024-04-25T01:43:17.418988+0000","last_became_peered":"2024-04-25T01:43:17.418988+0000","last_unstale":"2024-04-25T01:43:23.157305+0000","last_undegraded":"2024-04-25T01:43:23.157305+0000","last_fullsized":"2024-04-25T01:43:23.157305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:18.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158545+0000","last_change":"2024-04-25T01:43:17.430061+0000","last_active":"2024-04-25T01:43:23.158545+0000","last_peered":"2024-04-25T01:43:23.158545+0000","last_clean":"2024-04-25T01:43:23.158545+0000","last_became_active":"2024-04-25T01:43:17.429736+0000","last_became_peered":"2024-04-25T01:43:17.429736+0000","last_unstale":"2024-04-25T01:43:23.158545+0000","last_undegraded":"2024-04-25T01:43:23.158545+0000","last_fullsized":"2024-04-25T01:43:23.158545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:57.784221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721262+0000","last_change":"2024-04-25T01:43:17.422876+0000","last_active":"2024-04-25T01:43:22.721262+0000","last_peered":"2024-04-25T01:43:22.721262+0000","last_clean":"2024-04-25T01:43:22.721262+0000","last_became_active":"2024-04-25T01:43:17.422523+0000","last_became_peered":"2024-04-25T01:43:17.422523+0000","last_unstale":"2024-04-25T01:43:22.721262+0000","last_undegraded":"2024-04-25T01:43:22.721262+0000","last_fullsized":"2024-04-25T01:43:22.721262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:14.328021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.759026+0000","last_change":"2024-04-25T01:43:17.424941+0000","last_active":"2024-04-25T01:43:22.759026+0000","last_peered":"2024-04-25T01:43:22.759026+0000","last_clean":"2024-04-25T01:43:22.759026+0000","last_became_active":"2024-04-25T01:43:17.424620+0000","last_became_peered":"2024-04-25T01:43:17.424620+0000","last_unstale":"2024-04-25T01:43:22.759026+0000","last_undegraded":"2024-04-25T01:43:22.759026+0000","last_fullsized":"2024-04-25T01:43:22.759026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:06.111590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158253+0000","last_change":"2024-04-25T01:43:17.429188+0000","last_active":"2024-04-25T01:43:23.158253+0000","last_peered":"2024-04-25T01:43:23.158253+0000","last_clean":"2024-04-25T01:43:23.158253+0000","last_became_active":"2024-04-25T01:43:17.428837+0000","last_became_peered":"2024-04-25T01:43:17.428837+0000","last_unstale":"2024-04-25T01:43:23.158253+0000","last_undegraded":"2024-04-25T01:43:23.158253+0000","last_fullsized":"2024-04-25T01:43:23.158253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:08.923208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112431+0000","last_change":"2024-04-25T01:43:17.431402+0000","last_active":"2024-04-25T01:43:23.112431+0000","last_peered":"2024-04-25T01:43:23.112431+0000","last_clean":"2024-04-25T01:43:23.112431+0000","last_became_active":"2024-04-25T01:43:17.431044+0000","last_became_peered":"2024-04-25T01:43:17.431044+0000","last_unstale":"2024-04-25T01:43:23.112431+0000","last_undegraded":"2024-04-25T01:43:23.112431+0000","last_fullsized":"2024-04-25T01:43:23.112431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:20.731589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.430321+0000","last_change":"2024-04-25T01:43:17.430029+0000","last_active":"2024-04-25T01:43:22.430321+0000","last_peered":"2024-04-25T01:43:22.430321+0000","last_clean":"2024-04-25T01:43:22.430321+0000","last_became_active":"2024-04-25T01:43:17.428588+0000","last_became_peered":"2024-04-25T01:43:17.428588+0000","last_unstale":"2024-04-25T01:43:22.430321+0000","last_undegraded":"2024-04-25T01:43:22.430321+0000","last_fullsized":"2024-04-25T01:43:22.430321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:58.520175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.759364+0000","last_change":"2024-04-25T01:43:17.430377+0000","last_active":"2024-04-25T01:43:22.759364+0000","last_peered":"2024-04-25T01:43:22.759364+0000","last_clean":"2024-04-25T01:43:22.759364+0000","last_became_active":"2024-04-25T01:43:17.426609+0000","last_became_peered":"2024-04-25T01:43:17.426609+0000","last_unstale":"2024-04-25T01:43:22.759364+0000","last_undegraded":"2024-04-25T01:43:22.759364+0000","last_fullsized":"2024-04-25T01:43:22.759364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:10.696040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157957+0000","last_change":"2024-04-25T01:43:17.431561+0000","last_active":"2024-04-25T01:43:23.157957+0000","last_peered":"2024-04-25T01:43:23.157957+0000","last_clean":"2024-04-25T01:43:23.157957+0000","last_became_active":"2024-04-25T01:43:17.431229+0000","last_became_peered":"2024-04-25T01:43:17.431229+0000","last_unstale":"2024-04-25T01:43:23.157957+0000","last_undegraded":"2024-04-25T01:43:23.157957+0000","last_fullsized":"2024-04-25T01:43:23.157957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:03.686373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158407+0000","last_change":"2024-04-25T01:43:17.428442+0000","last_active":"2024-04-25T01:43:23.158407+0000","last_peered":"2024-04-25T01:43:23.158407+0000","last_clean":"2024-04-25T01:43:23.158407+0000","last_became_active":"2024-04-25T01:43:17.428052+0000","last_became_peered":"2024-04-25T01:43:17.428052+0000","last_unstale":"2024-04-25T01:43:23.158407+0000","last_undegraded":"2024-04-25T01:43:23.158407+0000","last_fullsized":"2024-04-25T01:43:23.158407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:06.097229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431003+0000","last_change":"2024-04-25T01:43:17.420520+0000","last_active":"2024-04-25T01:43:22.431003+0000","last_peered":"2024-04-25T01:43:22.431003+0000","last_clean":"2024-04-25T01:43:22.431003+0000","last_became_active":"2024-04-25T01:43:17.420171+0000","last_became_peered":"2024-04-25T01:43:17.420171+0000","last_unstale":"2024-04-25T01:43:22.431003+0000","last_undegraded":"2024-04-25T01:43:22.431003+0000","last_fullsized":"2024-04-25T01:43:22.431003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:43.699472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.156824+0000","last_change":"2024-04-25T01:43:17.422114+0000","last_active":"2024-04-25T01:43:23.156824+0000","last_peered":"2024-04-25T01:43:23.156824+0000","last_clean":"2024-04-25T01:43:23.156824+0000","last_became_active":"2024-04-25T01:43:17.421729+0000","last_became_peered":"2024-04-25T01:43:17.421729+0000","last_unstale":"2024-04-25T01:43:23.156824+0000","last_undegraded":"2024-04-25T01:43:23.156824+0000","last_fullsized":"2024-04-25T01:43:23.156824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:00.955123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.156797+0000","last_change":"2024-04-25T01:43:17.427344+0000","last_active":"2024-04-25T01:43:23.156797+0000","last_peered":"2024-04-25T01:43:23.156797+0000","last_clean":"2024-04-25T01:43:23.156797+0000","last_became_active":"2024-04-25T01:43:17.427044+0000","last_became_peered":"2024-04-25T01:43:17.427044+0000","last_unstale":"2024-04-25T01:43:23.156797+0000","last_undegraded":"2024-04-25T01:43:23.156797+0000","last_fullsized":"2024-04-25T01:43:23.156797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:24.564654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.429595+0000","last_change":"2024-04-25T01:43:17.427604+0000","last_active":"2024-04-25T01:43:22.429595+0000","last_peered":"2024-04-25T01:43:22.429595+0000","last_clean":"2024-04-25T01:43:22.429595+0000","last_became_active":"2024-04-25T01:43:17.427286+0000","last_became_peered":"2024-04-25T01:43:17.427286+0000","last_unstale":"2024-04-25T01:43:22.429595+0000","last_undegraded":"2024-04-25T01:43:22.429595+0000","last_fullsized":"2024-04-25T01:43:22.429595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:25.584150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158017+0000","last_change":"2024-04-25T01:43:17.423401+0000","last_active":"2024-04-25T01:43:23.158017+0000","last_peered":"2024-04-25T01:43:23.158017+0000","last_clean":"2024-04-25T01:43:23.158017+0000","last_became_active":"2024-04-25T01:43:17.422807+0000","last_became_peered":"2024-04-25T01:43:17.422807+0000","last_unstale":"2024-04-25T01:43:23.158017+0000","last_undegraded":"2024-04-25T01:43:23.158017+0000","last_fullsized":"2024-04-25T01:43:23.158017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:58.461242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157769+0000","last_change":"2024-04-25T01:43:17.422054+0000","last_active":"2024-04-25T01:43:23.157769+0000","last_peered":"2024-04-25T01:43:23.157769+0000","last_clean":"2024-04-25T01:43:23.157769+0000","last_became_active":"2024-04-25T01:43:17.421746+0000","last_became_peered":"2024-04-25T01:43:17.421746+0000","last_unstale":"2024-04-25T01:43:23.157769+0000","last_undegraded":"2024-04-25T01:43:23.157769+0000","last_fullsized":"2024-04-25T01:43:23.157769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:28.405852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158207+0000","last_change":"2024-04-25T01:43:17.429288+0000","last_active":"2024-04-25T01:43:23.158207+0000","last_peered":"2024-04-25T01:43:23.158207+0000","last_clean":"2024-04-25T01:43:23.158207+0000","last_became_active":"2024-04-25T01:43:17.427794+0000","last_became_peered":"2024-04-25T01:43:17.427794+0000","last_unstale":"2024-04-25T01:43:23.158207+0000","last_undegraded":"2024-04-25T01:43:23.158207+0000","last_fullsized":"2024-04-25T01:43:23.158207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:33.003361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157921+0000","last_change":"2024-04-25T01:43:21.438427+0000","last_active":"2024-04-25T01:43:23.157921+0000","last_peered":"2024-04-25T01:43:23.157921+0000","last_clean":"2024-04-25T01:43:23.157921+0000","last_became_active":"2024-04-25T01:43:21.438018+0000","last_became_peered":"2024-04-25T01:43:21.438018+0000","last_unstale":"2024-04-25T01:43:23.157921+0000","last_undegraded":"2024-04-25T01:43:23.157921+0000","last_fullsized":"2024-04-25T01:43:23.157921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:46.509437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157946+0000","last_change":"2024-04-25T01:43:17.031110+0000","last_active":"2024-04-25T01:43:23.157946+0000","last_peered":"2024-04-25T01:43:23.157946+0000","last_clean":"2024-04-25T01:43:23.157946+0000","last_became_active":"2024-04-25T01:43:17.030806+0000","last_became_peered":"2024-04-25T01:43:17.030806+0000","last_unstale":"2024-04-25T01:43:23.157946+0000","last_undegraded":"2024-04-25T01:43:23.157946+0000","last_fullsized":"2024-04-25T01:43:23.157946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:52:12.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112942+0000","last_change":"2024-04-25T01:43:17.428847+0000","last_active":"2024-04-25T01:43:23.112942+0000","last_peered":"2024-04-25T01:43:23.112942+0000","last_clean":"2024-04-25T01:43:23.112942+0000","last_became_active":"2024-04-25T01:43:17.428553+0000","last_became_peered":"2024-04-25T01:43:17.428553+0000","last_unstale":"2024-04-25T01:43:23.112942+0000","last_undegraded":"2024-04-25T01:43:23.112942+0000","last_fullsized":"2024-04-25T01:43:23.112942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:13.973275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157346+0000","last_change":"2024-04-25T01:43:21.452786+0000","last_active":"2024-04-25T01:43:23.157346+0000","last_peered":"2024-04-25T01:43:23.157346+0000","last_clean":"2024-04-25T01:43:23.157346+0000","last_became_active":"2024-04-25T01:43:21.451595+0000","last_became_peered":"2024-04-25T01:43:21.451595+0000","last_unstale":"2024-04-25T01:43:23.157346+0000","last_undegraded":"2024-04-25T01:43:23.157346+0000","last_fullsized":"2024-04-25T01:43:23.157346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:06.559390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.113136+0000","last_change":"2024-04-25T01:43:17.032954+0000","last_active":"2024-04-25T01:43:23.113136+0000","last_peered":"2024-04-25T01:43:23.113136+0000","last_clean":"2024-04-25T01:43:23.113136+0000","last_became_active":"2024-04-25T01:43:17.032627+0000","last_became_peered":"2024-04-25T01:43:17.032627+0000","last_unstale":"2024-04-25T01:43:23.113136+0000","last_undegraded":"2024-04-25T01:43:23.113136+0000","last_fullsized":"2024-04-25T01:43:23.113136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:07.167227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157128+0000","last_change":"2024-04-25T01:43:17.419451+0000","last_active":"2024-04-25T01:43:23.157128+0000","last_peered":"2024-04-25T01:43:23.157128+0000","last_clean":"2024-04-25T01:43:23.157128+0000","last_became_active":"2024-04-25T01:43:17.419111+0000","last_became_peered":"2024-04-25T01:43:17.419111+0000","last_unstale":"2024-04-25T01:43:23.157128+0000","last_undegraded":"2024-04-25T01:43:23.157128+0000","last_fullsized":"2024-04-25T01:43:23.157128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:41.284248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157863+0000","last_change":"2024-04-25T01:43:21.452525+0000","last_active":"2024-04-25T01:43:23.157863+0000","last_peered":"2024-04-25T01:43:23.157863+0000","last_clean":"2024-04-25T01:43:23.157863+0000","last_became_active":"2024-04-25T01:43:21.451834+0000","last_became_peered":"2024-04-25T01:43:21.451834+0000","last_unstale":"2024-04-25T01:43:23.157863+0000","last_undegraded":"2024-04-25T01:43:23.157863+0000","last_fullsized":"2024-04-25T01:43:23.157863+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:34.866314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721753+0000","last_change":"2024-04-25T01:43:17.035737+0000","last_active":"2024-04-25T01:43:22.721753+0000","last_peered":"2024-04-25T01:43:22.721753+0000","last_clean":"2024-04-25T01:43:22.721753+0000","last_became_active":"2024-04-25T01:43:17.034709+0000","last_became_peered":"2024-04-25T01:43:17.034709+0000","last_unstale":"2024-04-25T01:43:22.721753+0000","last_undegraded":"2024-04-25T01:43:22.721753+0000","last_fullsized":"2024-04-25T01:43:22.721753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:50.419193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.429986+0000","last_change":"2024-04-25T01:43:17.422530+0000","last_active":"2024-04-25T01:43:22.429986+0000","last_peered":"2024-04-25T01:43:22.429986+0000","last_clean":"2024-04-25T01:43:22.429986+0000","last_became_active":"2024-04-25T01:43:17.422197+0000","last_became_peered":"2024-04-25T01:43:17.422197+0000","last_unstale":"2024-04-25T01:43:22.429986+0000","last_undegraded":"2024-04-25T01:43:22.429986+0000","last_fullsized":"2024-04-25T01:43:22.429986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:25.250947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157384+0000","last_change":"2024-04-25T01:43:21.446869+0000","last_active":"2024-04-25T01:43:23.157384+0000","last_peered":"2024-04-25T01:43:23.157384+0000","last_clean":"2024-04-25T01:43:23.157384+0000","last_became_active":"2024-04-25T01:43:21.446446+0000","last_became_peered":"2024-04-25T01:43:21.446446+0000","last_unstale":"2024-04-25T01:43:23.157384+0000","last_undegraded":"2024-04-25T01:43:23.157384+0000","last_fullsized":"2024-04-25T01:43:23.157384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:21.023845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721751+0000","last_change":"2024-04-25T01:43:17.027614+0000","last_active":"2024-04-25T01:43:22.721751+0000","last_peered":"2024-04-25T01:43:22.721751+0000","last_clean":"2024-04-25T01:43:22.721751+0000","last_became_active":"2024-04-25T01:43:17.027235+0000","last_became_peered":"2024-04-25T01:43:17.027235+0000","last_unstale":"2024-04-25T01:43:22.721751+0000","last_undegraded":"2024-04-25T01:43:22.721751+0000","last_fullsized":"2024-04-25T01:43:22.721751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:45.627146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431928+0000","last_change":"2024-04-25T01:43:17.425813+0000","last_active":"2024-04-25T01:43:22.431928+0000","last_peered":"2024-04-25T01:43:22.431928+0000","last_clean":"2024-04-25T01:43:22.431928+0000","last_became_active":"2024-04-25T01:43:17.425469+0000","last_became_peered":"2024-04-25T01:43:17.425469+0000","last_unstale":"2024-04-25T01:43:22.431928+0000","last_undegraded":"2024-04-25T01:43:22.431928+0000","last_fullsized":"2024-04-25T01:43:22.431928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:49.863358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158138+0000","last_change":"2024-04-25T01:43:21.444557+0000","last_active":"2024-04-25T01:43:23.158138+0000","last_peered":"2024-04-25T01:43:23.158138+0000","last_clean":"2024-04-25T01:43:23.158138+0000","last_became_active":"2024-04-25T01:43:21.443809+0000","last_became_peered":"2024-04-25T01:43:21.443809+0000","last_unstale":"2024-04-25T01:43:23.158138+0000","last_undegraded":"2024-04-25T01:43:23.158138+0000","last_fullsized":"2024-04-25T01:43:23.158138+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:29.488448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:43.419368+0000","last_change":"2024-04-25T01:43:17.032280+0000","last_active":"2024-04-25T01:43:43.419368+0000","last_peered":"2024-04-25T01:43:43.419368+0000","last_clean":"2024-04-25T01:43:43.419368+0000","last_became_active":"2024-04-25T01:43:17.031649+0000","last_became_peered":"2024-04-25T01:43:17.031649+0000","last_unstale":"2024-04-25T01:43:43.419368+0000","last_undegraded":"2024-04-25T01:43:43.419368+0000","last_fullsized":"2024-04-25T01:43:43.419368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:08.532261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721299+0000","last_change":"2024-04-25T01:43:17.420350+0000","last_active":"2024-04-25T01:43:22.721299+0000","last_peered":"2024-04-25T01:43:22.721299+0000","last_clean":"2024-04-25T01:43:22.721299+0000","last_became_active":"2024-04-25T01:43:17.420024+0000","last_became_peered":"2024-04-25T01:43:17.420024+0000","last_unstale":"2024-04-25T01:43:22.721299+0000","last_undegraded":"2024-04-25T01:43:22.721299+0000","last_fullsized":"2024-04-25T01:43:22.721299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:40:19.395939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158271+0000","last_change":"2024-04-25T01:43:21.442683+0000","last_active":"2024-04-25T01:43:23.158271+0000","last_peered":"2024-04-25T01:43:23.158271+0000","last_clean":"2024-04-25T01:43:23.158271+0000","last_became_active":"2024-04-25T01:43:21.442222+0000","last_became_peered":"2024-04-25T01:43:21.442222+0000","last_unstale":"2024-04-25T01:43:23.158271+0000","last_undegraded":"2024-04-25T01:43:23.158271+0000","last_fullsized":"2024-04-25T01:43:23.158271+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:25.196600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.758997+0000","last_change":"2024-04-25T01:43:17.028054+0000","last_active":"2024-04-25T01:43:22.758997+0000","last_peered":"2024-04-25T01:43:22.758997+0000","last_clean":"2024-04-25T01:43:22.758997+0000","last_became_active":"2024-04-25T01:43:17.027639+0000","last_became_peered":"2024-04-25T01:43:17.027639+0000","last_unstale":"2024-04-25T01:43:22.758997+0000","last_undegraded":"2024-04-25T01:43:22.758997+0000","last_fullsized":"2024-04-25T01:43:22.758997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:40.569221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431758+0000","last_change":"2024-04-25T01:43:17.430417+0000","last_active":"2024-04-25T01:43:22.431758+0000","last_peered":"2024-04-25T01:43:22.431758+0000","last_clean":"2024-04-25T01:43:22.431758+0000","last_became_active":"2024-04-25T01:43:17.430024+0000","last_became_peered":"2024-04-25T01:43:17.430024+0000","last_unstale":"2024-04-25T01:43:22.431758+0000","last_undegraded":"2024-04-25T01:43:22.431758+0000","last_fullsized":"2024-04-25T01:43:22.431758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:56.834530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158064+0000","last_change":"2024-04-25T01:43:21.447732+0000","last_active":"2024-04-25T01:43:23.158064+0000","last_peered":"2024-04-25T01:43:23.158064+0000","last_clean":"2024-04-25T01:43:23.158064+0000","last_became_active":"2024-04-25T01:43:21.447315+0000","last_became_peered":"2024-04-25T01:43:21.447315+0000","last_unstale":"2024-04-25T01:43:23.158064+0000","last_undegraded":"2024-04-25T01:43:23.158064+0000","last_fullsized":"2024-04-25T01:43:23.158064+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:18.138828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.008657+0000","last_change":"2024-04-25T01:43:17.018769+0000","last_active":"2024-04-25T01:43:42.008657+0000","last_peered":"2024-04-25T01:43:42.008657+0000","last_clean":"2024-04-25T01:43:42.008657+0000","last_became_active":"2024-04-25T01:43:17.018169+0000","last_became_peered":"2024-04-25T01:43:17.018169+0000","last_unstale":"2024-04-25T01:43:42.008657+0000","last_undegraded":"2024-04-25T01:43:42.008657+0000","last_fullsized":"2024-04-25T01:43:42.008657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:59.112199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112568+0000","last_change":"2024-04-25T01:43:17.426289+0000","last_active":"2024-04-25T01:43:23.112568+0000","last_peered":"2024-04-25T01:43:23.112568+0000","last_clean":"2024-04-25T01:43:23.112568+0000","last_became_active":"2024-04-25T01:43:17.425512+0000","last_became_peered":"2024-04-25T01:43:17.425512+0000","last_unstale":"2024-04-25T01:43:23.112568+0000","last_undegraded":"2024-04-25T01:43:23.112568+0000","last_fullsized":"2024-04-25T01:43:23.112568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:47.956758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157217+0000","last_change":"2024-04-25T01:43:21.449131+0000","last_active":"2024-04-25T01:43:23.157217+0000","last_peered":"2024-04-25T01:43:23.157217+0000","last_clean":"2024-04-25T01:43:23.157217+0000","last_became_active":"2024-04-25T01:43:21.448752+0000","last_became_peered":"2024-04-25T01:43:21.448752+0000","last_unstale":"2024-04-25T01:43:23.157217+0000","last_undegraded":"2024-04-25T01:43:23.157217+0000","last_fullsized":"2024-04-25T01:43:23.157217+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:23.109582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.425543+0000","last_change":"2024-04-25T01:43:17.032414+0000","last_active":"2024-04-25T01:43:42.425543+0000","last_peered":"2024-04-25T01:43:42.425543+0000","last_clean":"2024-04-25T01:43:42.425543+0000","last_became_active":"2024-04-25T01:43:17.031848+0000","last_became_peered":"2024-04-25T01:43:17.031848+0000","last_unstale":"2024-04-25T01:43:42.425543+0000","last_undegraded":"2024-04-25T01:43:42.425543+0000","last_fullsized":"2024-04-25T01:43:42.425543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:37.034113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157484+0000","last_change":"2024-04-25T01:43:17.422831+0000","last_active":"2024-04-25T01:43:23.157484+0000","last_peered":"2024-04-25T01:43:23.157484+0000","last_clean":"2024-04-25T01:43:23.157484+0000","last_became_active":"2024-04-25T01:43:17.422238+0000","last_became_peered":"2024-04-25T01:43:17.422238+0000","last_unstale":"2024-04-25T01:43:23.157484+0000","last_undegraded":"2024-04-25T01:43:23.157484+0000","last_fullsized":"2024-04-25T01:43:23.157484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:05.740054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112547+0000","last_change":"2024-04-25T01:43:21.442268+0000","last_active":"2024-04-25T01:43:23.112547+0000","last_peered":"2024-04-25T01:43:23.112547+0000","last_clean":"2024-04-25T01:43:23.112547+0000","last_became_active":"2024-04-25T01:43:21.441844+0000","last_became_peered":"2024-04-25T01:43:21.441844+0000","last_unstale":"2024-04-25T01:43:23.112547+0000","last_undegraded":"2024-04-25T01:43:23.112547+0000","last_fullsized":"2024-04-25T01:43:23.112547+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:21.627312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:41.001380+0000","last_change":"2024-04-25T01:43:17.022576+0000","last_active":"2024-04-25T01:43:41.001380+0000","last_peered":"2024-04-25T01:43:41.001380+0000","last_clean":"2024-04-25T01:43:41.001380+0000","last_became_active":"2024-04-25T01:43:17.022223+0000","last_became_peered":"2024-04-25T01:43:17.022223+0000","last_unstale":"2024-04-25T01:43:41.001380+0000","last_undegraded":"2024-04-25T01:43:41.001380+0000","last_fullsized":"2024-04-25T01:43:41.001380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:23.905397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.430248+0000","last_change":"2024-04-25T01:43:17.421460+0000","last_active":"2024-04-25T01:43:22.430248+0000","last_peered":"2024-04-25T01:43:22.430248+0000","last_clean":"2024-04-25T01:43:22.430248+0000","last_became_active":"2024-04-25T01:43:17.421153+0000","last_became_peered":"2024-04-25T01:43:17.421153+0000","last_unstale":"2024-04-25T01:43:22.430248+0000","last_undegraded":"2024-04-25T01:43:22.430248+0000","last_fullsized":"2024-04-25T01:43:22.430248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:50.285476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.156968+0000","last_change":"2024-04-25T01:43:21.450961+0000","last_active":"2024-04-25T01:43:23.156968+0000","last_peered":"2024-04-25T01:43:23.156968+0000","last_clean":"2024-04-25T01:43:23.156968+0000","last_became_active":"2024-04-25T01:43:21.450578+0000","last_became_peered":"2024-04-25T01:43:21.450578+0000","last_unstale":"2024-04-25T01:43:23.156968+0000","last_undegraded":"2024-04-25T01:43:23.156968+0000","last_fullsized":"2024-04-25T01:43:23.156968+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:04.222443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:41.001075+0000","last_change":"2024-04-25T01:43:17.030973+0000","last_active":"2024-04-25T01:43:41.001075+0000","last_peered":"2024-04-25T01:43:41.001075+0000","last_clean":"2024-04-25T01:43:41.001075+0000","last_became_active":"2024-04-25T01:43:17.030458+0000","last_became_peered":"2024-04-25T01:43:17.030458+0000","last_unstale":"2024-04-25T01:43:41.001075+0000","last_undegraded":"2024-04-25T01:43:41.001075+0000","last_fullsized":"2024-04-25T01:43:41.001075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:19.187667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157097+0000","last_change":"2024-04-25T01:43:17.426748+0000","last_active":"2024-04-25T01:43:23.157097+0000","last_peered":"2024-04-25T01:43:23.157097+0000","last_clean":"2024-04-25T01:43:23.157097+0000","last_became_active":"2024-04-25T01:43:17.426447+0000","last_became_peered":"2024-04-25T01:43:17.426447+0000","last_unstale":"2024-04-25T01:43:23.157097+0000","last_undegraded":"2024-04-25T01:43:23.157097+0000","last_fullsized":"2024-04-25T01:43:23.157097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:39.794142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158014+0000","last_change":"2024-04-25T01:43:21.452287+0000","last_active":"2024-04-25T01:43:23.158014+0000","last_peered":"2024-04-25T01:43:23.158014+0000","last_clean":"2024-04-25T01:43:23.158014+0000","last_became_active":"2024-04-25T01:43:21.451765+0000","last_became_peered":"2024-04-25T01:43:21.451765+0000","last_unstale":"2024-04-25T01:43:23.158014+0000","last_undegraded":"2024-04-25T01:43:23.158014+0000","last_fullsized":"2024-04-25T01:43:23.158014+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:02.353645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.417131+0000","last_change":"2024-04-25T01:43:17.022319+0000","last_active":"2024-04-25T01:43:42.417131+0000","last_peered":"2024-04-25T01:43:42.417131+0000","last_clean":"2024-04-25T01:43:42.417131+0000","last_became_active":"2024-04-25T01:43:17.021944+0000","last_became_peered":"2024-04-25T01:43:17.021944+0000","last_unstale":"2024-04-25T01:43:42.417131+0000","last_undegraded":"2024-04-25T01:43:42.417131+0000","last_fullsized":"2024-04-25T01:43:42.417131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:42.856997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.430184+0000","last_change":"2024-04-25T01:43:17.418164+0000","last_active":"2024-04-25T01:43:22.430184+0000","last_peered":"2024-04-25T01:43:22.430184+0000","last_clean":"2024-04-25T01:43:22.430184+0000","last_became_active":"2024-04-25T01:43:17.417846+0000","last_became_peered":"2024-04-25T01:43:17.417846+0000","last_unstale":"2024-04-25T01:43:22.430184+0000","last_undegraded":"2024-04-25T01:43:22.430184+0000","last_fullsized":"2024-04-25T01:43:22.430184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:25.973457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157297+0000","last_change":"2024-04-25T01:43:21.443429+0000","last_active":"2024-04-25T01:43:23.157297+0000","last_peered":"2024-04-25T01:43:23.157297+0000","last_clean":"2024-04-25T01:43:23.157297+0000","last_became_active":"2024-04-25T01:43:21.442210+0000","last_became_peered":"2024-04-25T01:43:21.442210+0000","last_unstale":"2024-04-25T01:43:23.157297+0000","last_undegraded":"2024-04-25T01:43:23.157297+0000","last_fullsized":"2024-04-25T01:43:23.157297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:29.578740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'11","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:41.000319+0000","last_change":"2024-04-25T01:43:17.020501+0000","last_active":"2024-04-25T01:43:41.000319+0000","last_peered":"2024-04-25T01:43:41.000319+0000","last_clean":"2024-04-25T01:43:41.000319+0000","last_became_active":"2024-04-25T01:43:17.020160+0000","last_became_peered":"2024-04-25T01:43:17.020160+0000","last_unstale":"2024-04-25T01:43:41.000319+0000","last_undegraded":"2024-04-25T01:43:41.000319+0000","last_fullsized":"2024-04-25T01:43:41.000319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:49.273381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721533+0000","last_change":"2024-04-25T01:43:17.426895+0000","last_active":"2024-04-25T01:43:22.721533+0000","last_peered":"2024-04-25T01:43:22.721533+0000","last_clean":"2024-04-25T01:43:22.721533+0000","last_became_active":"2024-04-25T01:43:17.426572+0000","last_became_peered":"2024-04-25T01:43:17.426572+0000","last_unstale":"2024-04-25T01:43:22.721533+0000","last_undegraded":"2024-04-25T01:43:22.721533+0000","last_fullsized":"2024-04-25T01:43:22.721533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:36.166282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.432315+0000","last_change":"2024-04-25T01:43:21.442204+0000","last_active":"2024-04-25T01:43:22.432315+0000","last_peered":"2024-04-25T01:43:22.432315+0000","last_clean":"2024-04-25T01:43:22.432315+0000","last_became_active":"2024-04-25T01:43:21.441779+0000","last_became_peered":"2024-04-25T01:43:21.441779+0000","last_unstale":"2024-04-25T01:43:22.432315+0000","last_undegraded":"2024-04-25T01:43:22.432315+0000","last_fullsized":"2024-04-25T01:43:22.432315+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:12.075657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:43.442417+0000","last_change":"2024-04-25T01:43:17.028737+0000","last_active":"2024-04-25T01:43:43.442417+0000","last_peered":"2024-04-25T01:43:43.442417+0000","last_clean":"2024-04-25T01:43:43.442417+0000","last_became_active":"2024-04-25T01:43:17.028349+0000","last_became_peered":"2024-04-25T01:43:17.028349+0000","last_unstale":"2024-04-25T01:43:43.442417+0000","last_undegraded":"2024-04-25T01:43:43.442417+0000","last_fullsized":"2024-04-25T01:43:43.442417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:04.802232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431051+0000","last_change":"2024-04-25T01:43:17.436315+0000","last_active":"2024-04-25T01:43:22.431051+0000","last_peered":"2024-04-25T01:43:22.431051+0000","last_clean":"2024-04-25T01:43:22.431051+0000","last_became_active":"2024-04-25T01:43:17.436007+0000","last_became_peered":"2024-04-25T01:43:17.436007+0000","last_unstale":"2024-04-25T01:43:22.431051+0000","last_undegraded":"2024-04-25T01:43:22.431051+0000","last_fullsized":"2024-04-25T01:43:22.431051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:49.091092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157664+0000","last_change":"2024-04-25T01:43:21.446834+0000","last_active":"2024-04-25T01:43:23.157664+0000","last_peered":"2024-04-25T01:43:23.157664+0000","last_clean":"2024-04-25T01:43:23.157664+0000","last_became_active":"2024-04-25T01:43:21.446346+0000","last_became_peered":"2024-04-25T01:43:21.446346+0000","last_unstale":"2024-04-25T01:43:23.157664+0000","last_undegraded":"2024-04-25T01:43:23.157664+0000","last_fullsized":"2024-04-25T01:43:23.157664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:00.050981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.425322+0000","last_change":"2024-04-25T01:43:17.029513+0000","last_active":"2024-04-25T01:43:42.425322+0000","last_peered":"2024-04-25T01:43:42.425322+0000","last_clean":"2024-04-25T01:43:42.425322+0000","last_became_active":"2024-04-25T01:43:17.029203+0000","last_became_peered":"2024-04-25T01:43:17.029203+0000","last_unstale":"2024-04-25T01:43:42.425322+0000","last_undegraded":"2024-04-25T01:43:42.425322+0000","last_fullsized":"2024-04-25T01:43:42.425322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:47.002587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158090+0000","last_change":"2024-04-25T01:43:17.435273+0000","last_active":"2024-04-25T01:43:23.158090+0000","last_peered":"2024-04-25T01:43:23.158090+0000","last_clean":"2024-04-25T01:43:23.158090+0000","last_became_active":"2024-04-25T01:43:17.434892+0000","last_became_peered":"2024-04-25T01:43:17.434892+0000","last_unstale":"2024-04-25T01:43:23.158090+0000","last_undegraded":"2024-04-25T01:43:23.158090+0000","last_fullsized":"2024-04-25T01:43:23.158090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:55.342440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157761+0000","last_change":"2024-04-25T01:43:21.448363+0000","last_active":"2024-04-25T01:43:23.157761+0000","last_peered":"2024-04-25T01:43:23.157761+0000","last_clean":"2024-04-25T01:43:23.157761+0000","last_became_active":"2024-04-25T01:43:21.445980+0000","last_became_peered":"2024-04-25T01:43:21.445980+0000","last_unstale":"2024-04-25T01:43:23.157761+0000","last_undegraded":"2024-04-25T01:43:23.157761+0000","last_fullsized":"2024-04-25T01:43:23.157761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:05.890579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:43.422804+0000","last_change":"2024-04-25T01:43:17.028557+0000","last_active":"2024-04-25T01:43:43.422804+0000","last_peered":"2024-04-25T01:43:43.422804+0000","last_clean":"2024-04-25T01:43:43.422804+0000","last_became_active":"2024-04-25T01:43:17.028199+0000","last_became_peered":"2024-04-25T01:43:17.028199+0000","last_unstale":"2024-04-25T01:43:43.422804+0000","last_undegraded":"2024-04-25T01:43:43.422804+0000","last_fullsized":"2024-04-25T01:43:43.422804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:41.809220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157293+0000","last_change":"2024-04-25T01:43:17.437002+0000","last_active":"2024-04-25T01:43:23.157293+0000","last_peered":"2024-04-25T01:43:23.157293+0000","last_clean":"2024-04-25T01:43:23.157293+0000","last_became_active":"2024-04-25T01:43:17.436509+0000","last_became_peered":"2024-04-25T01:43:17.436509+0000","last_unstale":"2024-04-25T01:43:23.157293+0000","last_undegraded":"2024-04-25T01:43:23.157293+0000","last_fullsized":"2024-04-25T01:43:23.157293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:49.329249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431432+0000","last_change":"2024-04-25T01:43:21.447748+0000","last_active":"2024-04-25T01:43:22.431432+0000","last_peered":"2024-04-25T01:43:22.431432+0000","last_clean":"2024-04-25T01:43:22.431432+0000","last_became_active":"2024-04-25T01:43:21.447368+0000","last_became_peered":"2024-04-25T01:43:21.447368+0000","last_unstale":"2024-04-25T01:43:22.431432+0000","last_undegraded":"2024-04-25T01:43:22.431432+0000","last_fullsized":"2024-04-25T01:43:22.431432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:53.506298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721191+0000","last_change":"2024-04-25T01:43:17.028185+0000","last_active":"2024-04-25T01:43:22.721191+0000","last_peered":"2024-04-25T01:43:22.721191+0000","last_clean":"2024-04-25T01:43:22.721191+0000","last_became_active":"2024-04-25T01:43:17.027851+0000","last_became_peered":"2024-04-25T01:43:17.027851+0000","last_unstale":"2024-04-25T01:43:22.721191+0000","last_undegraded":"2024-04-25T01:43:22.721191+0000","last_fullsized":"2024-04-25T01:43:22.721191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:02.320793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431342+0000","last_change":"2024-04-25T01:43:17.424990+0000","last_active":"2024-04-25T01:43:22.431342+0000","last_peered":"2024-04-25T01:43:22.431342+0000","last_clean":"2024-04-25T01:43:22.431342+0000","last_became_active":"2024-04-25T01:43:17.424598+0000","last_became_peered":"2024-04-25T01:43:17.424598+0000","last_unstale":"2024-04-25T01:43:22.431342+0000","last_undegraded":"2024-04-25T01:43:22.431342+0000","last_fullsized":"2024-04-25T01:43:22.431342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:34.855089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431511+0000","last_change":"2024-04-25T01:43:21.444138+0000","last_active":"2024-04-25T01:43:22.431511+0000","last_peered":"2024-04-25T01:43:22.431511+0000","last_clean":"2024-04-25T01:43:22.431511+0000","last_became_active":"2024-04-25T01:43:21.443283+0000","last_became_peered":"2024-04-25T01:43:21.443283+0000","last_unstale":"2024-04-25T01:43:22.431511+0000","last_undegraded":"2024-04-25T01:43:22.431511+0000","last_fullsized":"2024-04-25T01:43:22.431511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:46.275789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721617+0000","last_change":"2024-04-25T01:43:17.027892+0000","last_active":"2024-04-25T01:43:22.721617+0000","last_peered":"2024-04-25T01:43:22.721617+0000","last_clean":"2024-04-25T01:43:22.721617+0000","last_became_active":"2024-04-25T01:43:17.027476+0000","last_became_peered":"2024-04-25T01:43:17.027476+0000","last_unstale":"2024-04-25T01:43:22.721617+0000","last_undegraded":"2024-04-25T01:43:22.721617+0000","last_fullsized":"2024-04-25T01:43:22.721617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:31.079700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157897+0000","last_change":"2024-04-25T01:43:17.431491+0000","last_active":"2024-04-25T01:43:23.157897+0000","last_peered":"2024-04-25T01:43:23.157897+0000","last_clean":"2024-04-25T01:43:23.157897+0000","last_became_active":"2024-04-25T01:43:17.431160+0000","last_became_peered":"2024-04-25T01:43:17.431160+0000","last_unstale":"2024-04-25T01:43:23.157897+0000","last_undegraded":"2024-04-25T01:43:23.157897+0000","last_fullsized":"2024-04-25T01:43:23.157897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:03.612857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721409+0000","last_change":"2024-04-25T01:43:21.626398+0000","last_active":"2024-04-25T01:43:22.721409+0000","last_peered":"2024-04-25T01:43:22.721409+0000","last_clean":"2024-04-25T01:43:22.721409+0000","last_became_active":"2024-04-25T01:43:21.625856+0000","last_became_peered":"2024-04-25T01:43:21.625856+0000","last_unstale":"2024-04-25T01:43:22.721409+0000","last_undegraded":"2024-04-25T01:43:22.721409+0000","last_fullsized":"2024-04-25T01:43:22.721409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:53.000912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157615+0000","last_change":"2024-04-25T01:43:17.027564+0000","last_active":"2024-04-25T01:43:23.157615+0000","last_peered":"2024-04-25T01:43:23.157615+0000","last_clean":"2024-04-25T01:43:23.157615+0000","last_became_active":"2024-04-25T01:43:17.027218+0000","last_became_peered":"2024-04-25T01:43:17.027218+0000","last_unstale":"2024-04-25T01:43:23.157615+0000","last_undegraded":"2024-04-25T01:43:23.157615+0000","last_fullsized":"2024-04-25T01:43:23.157615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:52.658023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.759150+0000","last_change":"2024-04-25T01:43:17.431647+0000","last_active":"2024-04-25T01:43:22.759150+0000","last_peered":"2024-04-25T01:43:22.759150+0000","last_clean":"2024-04-25T01:43:22.759150+0000","last_became_active":"2024-04-25T01:43:17.431048+0000","last_became_peered":"2024-04-25T01:43:17.431048+0000","last_unstale":"2024-04-25T01:43:22.759150+0000","last_undegraded":"2024-04-25T01:43:22.759150+0000","last_fullsized":"2024-04-25T01:43:22.759150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:30.124585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431037+0000","last_change":"2024-04-25T01:43:21.450326+0000","last_active":"2024-04-25T01:43:22.431037+0000","last_peered":"2024-04-25T01:43:22.431037+0000","last_clean":"2024-04-25T01:43:22.431037+0000","last_became_active":"2024-04-25T01:43:21.449925+0000","last_became_peered":"2024-04-25T01:43:21.449925+0000","last_unstale":"2024-04-25T01:43:22.431037+0000","last_undegraded":"2024-04-25T01:43:22.431037+0000","last_fullsized":"2024-04-25T01:43:22.431037+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:39.896905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157771+0000","last_change":"2024-04-25T01:43:17.032223+0000","last_active":"2024-04-25T01:43:23.157771+0000","last_peered":"2024-04-25T01:43:23.157771+0000","last_clean":"2024-04-25T01:43:23.157771+0000","last_became_active":"2024-04-25T01:43:17.031774+0000","last_became_peered":"2024-04-25T01:43:17.031774+0000","last_unstale":"2024-04-25T01:43:23.157771+0000","last_undegraded":"2024-04-25T01:43:23.157771+0000","last_fullsized":"2024-04-25T01:43:23.157771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:03.730850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.759222+0000","last_change":"2024-04-25T01:43:17.425756+0000","last_active":"2024-04-25T01:43:22.759222+0000","last_peered":"2024-04-25T01:43:22.759222+0000","last_clean":"2024-04-25T01:43:22.759222+0000","last_became_active":"2024-04-25T01:43:17.425433+0000","last_became_peered":"2024-04-25T01:43:17.425433+0000","last_unstale":"2024-04-25T01:43:22.759222+0000","last_undegraded":"2024-04-25T01:43:22.759222+0000","last_fullsized":"2024-04-25T01:43:22.759222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:29.776771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431289+0000","last_change":"2024-04-25T01:43:21.625898+0000","last_active":"2024-04-25T01:43:22.431289+0000","last_peered":"2024-04-25T01:43:22.431289+0000","last_clean":"2024-04-25T01:43:22.431289+0000","last_became_active":"2024-04-25T01:43:21.625460+0000","last_became_peered":"2024-04-25T01:43:21.625460+0000","last_unstale":"2024-04-25T01:43:22.431289+0000","last_undegraded":"2024-04-25T01:43:22.431289+0000","last_fullsized":"2024-04-25T01:43:22.431289+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:31.199172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431720+0000","last_change":"2024-04-25T01:43:17.424709+0000","last_active":"2024-04-25T01:43:22.431720+0000","last_peered":"2024-04-25T01:43:22.431720+0000","last_clean":"2024-04-25T01:43:22.431720+0000","last_became_active":"2024-04-25T01:43:17.424364+0000","last_became_peered":"2024-04-25T01:43:17.424364+0000","last_unstale":"2024-04-25T01:43:22.431720+0000","last_undegraded":"2024-04-25T01:43:22.431720+0000","last_fullsized":"2024-04-25T01:43:22.431720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:16.699513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:43.422151+0000","last_change":"2024-04-25T01:43:17.028904+0000","last_active":"2024-04-25T01:43:43.422151+0000","last_peered":"2024-04-25T01:43:43.422151+0000","last_clean":"2024-04-25T01:43:43.422151+0000","last_became_active":"2024-04-25T01:43:17.028608+0000","last_became_peered":"2024-04-25T01:43:17.028608+0000","last_unstale":"2024-04-25T01:43:43.422151+0000","last_undegraded":"2024-04-25T01:43:43.422151+0000","last_fullsized":"2024-04-25T01:43:43.422151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:19.297926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157882+0000","last_change":"2024-04-25T01:43:21.440009+0000","last_active":"2024-04-25T01:43:23.157882+0000","last_peered":"2024-04-25T01:43:23.157882+0000","last_clean":"2024-04-25T01:43:23.157882+0000","last_became_active":"2024-04-25T01:43:21.439611+0000","last_became_peered":"2024-04-25T01:43:21.439611+0000","last_unstale":"2024-04-25T01:43:23.157882+0000","last_undegraded":"2024-04-25T01:43:23.157882+0000","last_fullsized":"2024-04-25T01:43:23.157882+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:14.952044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112678+0000","last_change":"2024-04-25T01:43:17.428636+0000","last_active":"2024-04-25T01:43:23.112678+0000","last_peered":"2024-04-25T01:43:23.112678+0000","last_clean":"2024-04-25T01:43:23.112678+0000","last_became_active":"2024-04-25T01:43:17.428316+0000","last_became_peered":"2024-04-25T01:43:17.428316+0000","last_unstale":"2024-04-25T01:43:23.112678+0000","last_undegraded":"2024-04-25T01:43:23.112678+0000","last_fullsized":"2024-04-25T01:43:23.112678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:04.683846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.424045+0000","last_change":"2024-04-25T01:43:17.020632+0000","last_active":"2024-04-25T01:43:42.424045+0000","last_peered":"2024-04-25T01:43:42.424045+0000","last_clean":"2024-04-25T01:43:42.424045+0000","last_became_active":"2024-04-25T01:43:17.020278+0000","last_became_peered":"2024-04-25T01:43:17.020278+0000","last_unstale":"2024-04-25T01:43:42.424045+0000","last_undegraded":"2024-04-25T01:43:42.424045+0000","last_fullsized":"2024-04-25T01:43:42.424045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:13.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":347,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157176+0000","last_change":"2024-04-25T01:42:05.003964+0000","last_active":"2024-04-25T01:43:23.157176+0000","last_peered":"2024-04-25T01:43:23.157176+0000","last_clean":"2024-04-25T01:43:23.157176+0000","last_became_active":"2024-04-25T01:42:05.003509+0000","last_became_peered":"2024-04-25T01:42:05.003509+0000","last_unstale":"2024-04-25T01:43:23.157176+0000","last_undegraded":"2024-04-25T01:43:23.157176+0000","last_fullsized":"2024-04-25T01:43:23.157176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:38:39.137002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:38:39.137002+0000","last_clean_scrub_stamp":"2024-04-25T01:38:39.137002+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:53.878838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.432168+0000","last_change":"2024-04-25T01:43:21.626345+0000","last_active":"2024-04-25T01:43:22.432168+0000","last_peered":"2024-04-25T01:43:22.432168+0000","last_clean":"2024-04-25T01:43:22.432168+0000","last_became_active":"2024-04-25T01:43:21.625884+0000","last_became_peered":"2024-04-25T01:43:21.625884+0000","last_unstale":"2024-04-25T01:43:22.432168+0000","last_undegraded":"2024-04-25T01:43:22.432168+0000","last_fullsized":"2024-04-25T01:43:22.432168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:37.961617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.430581+0000","last_change":"2024-04-25T01:43:17.421186+0000","last_active":"2024-04-25T01:43:22.430581+0000","last_peered":"2024-04-25T01:43:22.430581+0000","last_clean":"2024-04-25T01:43:22.430581+0000","last_became_active":"2024-04-25T01:43:17.420824+0000","last_became_peered":"2024-04-25T01:43:17.420824+0000","last_unstale":"2024-04-25T01:43:22.430581+0000","last_undegraded":"2024-04-25T01:43:22.430581+0000","last_fullsized":"2024-04-25T01:43:22.430581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:27.829126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.006961+0000","last_change":"2024-04-25T01:43:17.034050+0000","last_active":"2024-04-25T01:43:42.006961+0000","last_peered":"2024-04-25T01:43:42.006961+0000","last_clean":"2024-04-25T01:43:42.006961+0000","last_became_active":"2024-04-25T01:43:17.033725+0000","last_became_peered":"2024-04-25T01:43:17.033725+0000","last_unstale":"2024-04-25T01:43:42.006961+0000","last_undegraded":"2024-04-25T01:43:42.006961+0000","last_fullsized":"2024-04-25T01:43:42.006961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:06.419265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157725+0000","last_change":"2024-04-25T01:43:21.443299+0000","last_active":"2024-04-25T01:43:23.157725+0000","last_peered":"2024-04-25T01:43:23.157725+0000","last_clean":"2024-04-25T01:43:23.157725+0000","last_became_active":"2024-04-25T01:43:21.442899+0000","last_became_peered":"2024-04-25T01:43:21.442899+0000","last_unstale":"2024-04-25T01:43:23.157725+0000","last_undegraded":"2024-04-25T01:43:23.157725+0000","last_fullsized":"2024-04-25T01:43:23.157725+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:51.720208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721602+0000","last_change":"2024-04-25T01:43:17.035136+0000","last_active":"2024-04-25T01:43:22.721602+0000","last_peered":"2024-04-25T01:43:22.721602+0000","last_clean":"2024-04-25T01:43:22.721602+0000","last_became_active":"2024-04-25T01:43:17.034289+0000","last_became_peered":"2024-04-25T01:43:17.034289+0000","last_unstale":"2024-04-25T01:43:22.721602+0000","last_undegraded":"2024-04-25T01:43:22.721602+0000","last_fullsized":"2024-04-25T01:43:22.721602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:07.450025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431092+0000","last_change":"2024-04-25T01:43:17.414478+0000","last_active":"2024-04-25T01:43:22.431092+0000","last_peered":"2024-04-25T01:43:22.431092+0000","last_clean":"2024-04-25T01:43:22.431092+0000","last_became_active":"2024-04-25T01:43:17.414061+0000","last_became_peered":"2024-04-25T01:43:17.414061+0000","last_unstale":"2024-04-25T01:43:22.431092+0000","last_undegraded":"2024-04-25T01:43:22.431092+0000","last_fullsized":"2024-04-25T01:43:22.431092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:16.051518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112618+0000","last_change":"2024-04-25T01:43:21.625963+0000","last_active":"2024-04-25T01:43:23.112618+0000","last_peered":"2024-04-25T01:43:23.112618+0000","last_clean":"2024-04-25T01:43:23.112618+0000","last_became_active":"2024-04-25T01:43:21.625563+0000","last_became_peered":"2024-04-25T01:43:21.625563+0000","last_unstale":"2024-04-25T01:43:23.112618+0000","last_undegraded":"2024-04-25T01:43:23.112618+0000","last_fullsized":"2024-04-25T01:43:23.112618+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:11.217504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157738+0000","last_change":"2024-04-25T01:43:17.024391+0000","last_active":"2024-04-25T01:43:23.157738+0000","last_peered":"2024-04-25T01:43:23.157738+0000","last_clean":"2024-04-25T01:43:23.157738+0000","last_became_active":"2024-04-25T01:43:17.023994+0000","last_became_peered":"2024-04-25T01:43:17.023994+0000","last_unstale":"2024-04-25T01:43:23.157738+0000","last_undegraded":"2024-04-25T01:43:23.157738+0000","last_fullsized":"2024-04-25T01:43:23.157738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:56.489383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.113025+0000","last_change":"2024-04-25T01:43:17.435357+0000","last_active":"2024-04-25T01:43:23.113025+0000","last_peered":"2024-04-25T01:43:23.113025+0000","last_clean":"2024-04-25T01:43:23.113025+0000","last_became_active":"2024-04-25T01:43:17.435090+0000","last_became_peered":"2024-04-25T01:43:17.435090+0000","last_unstale":"2024-04-25T01:43:23.113025+0000","last_undegraded":"2024-04-25T01:43:23.113025+0000","last_fullsized":"2024-04-25T01:43:23.113025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:49.181850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158097+0000","last_change":"2024-04-25T01:43:21.441314+0000","last_active":"2024-04-25T01:43:23.158097+0000","last_peered":"2024-04-25T01:43:23.158097+0000","last_clean":"2024-04-25T01:43:23.158097+0000","last_became_active":"2024-04-25T01:43:21.440896+0000","last_became_peered":"2024-04-25T01:43:21.440896+0000","last_unstale":"2024-04-25T01:43:23.158097+0000","last_undegraded":"2024-04-25T01:43:23.158097+0000","last_fullsized":"2024-04-25T01:43:23.158097+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:36.091128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:43.440878+0000","last_change":"2024-04-25T01:43:17.031525+0000","last_active":"2024-04-25T01:43:43.440878+0000","last_peered":"2024-04-25T01:43:43.440878+0000","last_clean":"2024-04-25T01:43:43.440878+0000","last_became_active":"2024-04-25T01:43:17.031203+0000","last_became_peered":"2024-04-25T01:43:17.031203+0000","last_unstale":"2024-04-25T01:43:43.440878+0000","last_undegraded":"2024-04-25T01:43:43.440878+0000","last_fullsized":"2024-04-25T01:43:43.440878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:22.929179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157329+0000","last_change":"2024-04-25T01:43:17.425981+0000","last_active":"2024-04-25T01:43:23.157329+0000","last_peered":"2024-04-25T01:43:23.157329+0000","last_clean":"2024-04-25T01:43:23.157329+0000","last_became_active":"2024-04-25T01:43:17.425674+0000","last_became_peered":"2024-04-25T01:43:17.425674+0000","last_unstale":"2024-04-25T01:43:23.157329+0000","last_undegraded":"2024-04-25T01:43:23.157329+0000","last_fullsized":"2024-04-25T01:43:23.157329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:58.441774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158326+0000","last_change":"2024-04-25T01:43:21.626389+0000","last_active":"2024-04-25T01:43:23.158326+0000","last_peered":"2024-04-25T01:43:23.158326+0000","last_clean":"2024-04-25T01:43:23.158326+0000","last_became_active":"2024-04-25T01:43:21.625859+0000","last_became_peered":"2024-04-25T01:43:21.625859+0000","last_unstale":"2024-04-25T01:43:23.158326+0000","last_undegraded":"2024-04-25T01:43:23.158326+0000","last_fullsized":"2024-04-25T01:43:23.158326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:51.372126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.429851+0000","last_change":"2024-04-25T01:43:17.022903+0000","last_active":"2024-04-25T01:43:22.429851+0000","last_peered":"2024-04-25T01:43:22.429851+0000","last_clean":"2024-04-25T01:43:22.429851+0000","last_became_active":"2024-04-25T01:43:17.022564+0000","last_became_peered":"2024-04-25T01:43:17.022564+0000","last_unstale":"2024-04-25T01:43:22.429851+0000","last_undegraded":"2024-04-25T01:43:22.429851+0000","last_fullsized":"2024-04-25T01:43:22.429851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:01.277500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721183+0000","last_change":"2024-04-25T01:43:17.422855+0000","last_active":"2024-04-25T01:43:22.721183+0000","last_peered":"2024-04-25T01:43:22.721183+0000","last_clean":"2024-04-25T01:43:22.721183+0000","last_became_active":"2024-04-25T01:43:17.422490+0000","last_became_peered":"2024-04-25T01:43:17.422490+0000","last_unstale":"2024-04-25T01:43:22.721183+0000","last_undegraded":"2024-04-25T01:43:22.721183+0000","last_fullsized":"2024-04-25T01:43:22.721183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:40.016044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157958+0000","last_change":"2024-04-25T01:43:21.440080+0000","last_active":"2024-04-25T01:43:23.157958+0000","last_peered":"2024-04-25T01:43:23.157958+0000","last_clean":"2024-04-25T01:43:23.157958+0000","last_became_active":"2024-04-25T01:43:21.439669+0000","last_became_peered":"2024-04-25T01:43:21.439669+0000","last_unstale":"2024-04-25T01:43:23.157958+0000","last_undegraded":"2024-04-25T01:43:23.157958+0000","last_fullsized":"2024-04-25T01:43:23.157958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:34.143210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.423564+0000","last_change":"2024-04-25T01:43:17.035521+0000","last_active":"2024-04-25T01:43:42.423564+0000","last_peered":"2024-04-25T01:43:42.423564+0000","last_clean":"2024-04-25T01:43:42.423564+0000","last_became_active":"2024-04-25T01:43:17.033762+0000","last_became_peered":"2024-04-25T01:43:17.033762+0000","last_unstale":"2024-04-25T01:43:42.423564+0000","last_undegraded":"2024-04-25T01:43:42.423564+0000","last_fullsized":"2024-04-25T01:43:42.423564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:04.762578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157415+0000","last_change":"2024-04-25T01:43:17.426952+0000","last_active":"2024-04-25T01:43:23.157415+0000","last_peered":"2024-04-25T01:43:23.157415+0000","last_clean":"2024-04-25T01:43:23.157415+0000","last_became_active":"2024-04-25T01:43:17.426573+0000","last_became_peered":"2024-04-25T01:43:17.426573+0000","last_unstale":"2024-04-25T01:43:23.157415+0000","last_undegraded":"2024-04-25T01:43:23.157415+0000","last_fullsized":"2024-04-25T01:43:23.157415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:33.807254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158247+0000","last_change":"2024-04-25T01:43:21.453295+0000","last_active":"2024-04-25T01:43:23.158247+0000","last_peered":"2024-04-25T01:43:23.158247+0000","last_clean":"2024-04-25T01:43:23.158247+0000","last_became_active":"2024-04-25T01:43:21.452031+0000","last_became_peered":"2024-04-25T01:43:21.452031+0000","last_unstale":"2024-04-25T01:43:23.158247+0000","last_undegraded":"2024-04-25T01:43:23.158247+0000","last_fullsized":"2024-04-25T01:43:23.158247+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:23.956638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:41.995947+0000","last_change":"2024-04-25T01:43:17.029253+0000","last_active":"2024-04-25T01:43:41.995947+0000","last_peered":"2024-04-25T01:43:41.995947+0000","last_clean":"2024-04-25T01:43:41.995947+0000","last_became_active":"2024-04-25T01:43:17.028949+0000","last_became_peered":"2024-04-25T01:43:17.028949+0000","last_unstale":"2024-04-25T01:43:41.995947+0000","last_undegraded":"2024-04-25T01:43:41.995947+0000","last_fullsized":"2024-04-25T01:43:41.995947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:05.101916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112744+0000","last_change":"2024-04-25T01:43:17.431319+0000","last_active":"2024-04-25T01:43:23.112744+0000","last_peered":"2024-04-25T01:43:23.112744+0000","last_clean":"2024-04-25T01:43:23.112744+0000","last_became_active":"2024-04-25T01:43:17.431007+0000","last_became_peered":"2024-04-25T01:43:17.431007+0000","last_unstale":"2024-04-25T01:43:23.112744+0000","last_undegraded":"2024-04-25T01:43:23.112744+0000","last_fullsized":"2024-04-25T01:43:23.112744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:38.999286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158416+0000","last_change":"2024-04-25T01:43:21.625931+0000","last_active":"2024-04-25T01:43:23.158416+0000","last_peered":"2024-04-25T01:43:23.158416+0000","last_clean":"2024-04-25T01:43:23.158416+0000","last_became_active":"2024-04-25T01:43:21.625460+0000","last_became_peered":"2024-04-25T01:43:21.625460+0000","last_unstale":"2024-04-25T01:43:23.158416+0000","last_undegraded":"2024-04-25T01:43:23.158416+0000","last_fullsized":"2024-04-25T01:43:23.158416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:59.665909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.721268+0000","last_change":"2024-04-25T01:43:17.033937+0000","last_active":"2024-04-25T01:43:22.721268+0000","last_peered":"2024-04-25T01:43:22.721268+0000","last_clean":"2024-04-25T01:43:22.721268+0000","last_became_active":"2024-04-25T01:43:17.033629+0000","last_became_peered":"2024-04-25T01:43:17.033629+0000","last_unstale":"2024-04-25T01:43:22.721268+0000","last_undegraded":"2024-04-25T01:43:22.721268+0000","last_fullsized":"2024-04-25T01:43:22.721268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:05.531624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157856+0000","last_change":"2024-04-25T01:43:17.436068+0000","last_active":"2024-04-25T01:43:23.157856+0000","last_peered":"2024-04-25T01:43:23.157856+0000","last_clean":"2024-04-25T01:43:23.157856+0000","last_became_active":"2024-04-25T01:43:17.435383+0000","last_became_peered":"2024-04-25T01:43:17.435383+0000","last_unstale":"2024-04-25T01:43:23.157856+0000","last_undegraded":"2024-04-25T01:43:23.157856+0000","last_fullsized":"2024-04-25T01:43:23.157856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:00.087923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.429770+0000","last_change":"2024-04-25T01:43:21.441696+0000","last_active":"2024-04-25T01:43:22.429770+0000","last_peered":"2024-04-25T01:43:22.429770+0000","last_clean":"2024-04-25T01:43:22.429770+0000","last_became_active":"2024-04-25T01:43:21.441263+0000","last_became_peered":"2024-04-25T01:43:21.441263+0000","last_unstale":"2024-04-25T01:43:22.429770+0000","last_undegraded":"2024-04-25T01:43:22.429770+0000","last_fullsized":"2024-04-25T01:43:22.429770+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:57.263465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.427281+0000","last_change":"2024-04-25T01:43:17.033841+0000","last_active":"2024-04-25T01:43:42.427281+0000","last_peered":"2024-04-25T01:43:42.427281+0000","last_clean":"2024-04-25T01:43:42.427281+0000","last_became_active":"2024-04-25T01:43:17.033477+0000","last_became_peered":"2024-04-25T01:43:17.033477+0000","last_unstale":"2024-04-25T01:43:42.427281+0000","last_undegraded":"2024-04-25T01:43:42.427281+0000","last_fullsized":"2024-04-25T01:43:42.427281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:43.717525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158025+0000","last_change":"2024-04-25T01:43:17.430744+0000","last_active":"2024-04-25T01:43:23.158025+0000","last_peered":"2024-04-25T01:43:23.158025+0000","last_clean":"2024-04-25T01:43:23.158025+0000","last_became_active":"2024-04-25T01:43:17.430434+0000","last_became_peered":"2024-04-25T01:43:17.430434+0000","last_unstale":"2024-04-25T01:43:23.158025+0000","last_undegraded":"2024-04-25T01:43:23.158025+0000","last_fullsized":"2024-04-25T01:43:23.158025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:17.895901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.156878+0000","last_change":"2024-04-25T01:43:21.451604+0000","last_active":"2024-04-25T01:43:23.156878+0000","last_peered":"2024-04-25T01:43:23.156878+0000","last_clean":"2024-04-25T01:43:23.156878+0000","last_became_active":"2024-04-25T01:43:21.451178+0000","last_became_peered":"2024-04-25T01:43:21.451178+0000","last_unstale":"2024-04-25T01:43:23.156878+0000","last_undegraded":"2024-04-25T01:43:23.156878+0000","last_fullsized":"2024-04-25T01:43:23.156878+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:17.387498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:42.423954+0000","last_change":"2024-04-25T01:43:17.025580+0000","last_active":"2024-04-25T01:43:42.423954+0000","last_peered":"2024-04-25T01:43:42.423954+0000","last_clean":"2024-04-25T01:43:42.423954+0000","last_became_active":"2024-04-25T01:43:17.025267+0000","last_became_peered":"2024-04-25T01:43:17.025267+0000","last_unstale":"2024-04-25T01:43:42.423954+0000","last_undegraded":"2024-04-25T01:43:42.423954+0000","last_fullsized":"2024-04-25T01:43:42.423954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:04.922572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157533+0000","last_change":"2024-04-25T01:43:17.427502+0000","last_active":"2024-04-25T01:43:23.157533+0000","last_peered":"2024-04-25T01:43:23.157533+0000","last_clean":"2024-04-25T01:43:23.157533+0000","last_became_active":"2024-04-25T01:43:17.427149+0000","last_became_peered":"2024-04-25T01:43:17.427149+0000","last_unstale":"2024-04-25T01:43:23.157533+0000","last_undegraded":"2024-04-25T01:43:23.157533+0000","last_fullsized":"2024-04-25T01:43:23.157533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:18:49.292709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431300+0000","last_change":"2024-04-25T01:43:21.437988+0000","last_active":"2024-04-25T01:43:22.431300+0000","last_peered":"2024-04-25T01:43:22.431300+0000","last_clean":"2024-04-25T01:43:22.431300+0000","last_became_active":"2024-04-25T01:43:21.437523+0000","last_became_peered":"2024-04-25T01:43:21.437523+0000","last_unstale":"2024-04-25T01:43:22.431300+0000","last_undegraded":"2024-04-25T01:43:22.431300+0000","last_fullsized":"2024-04-25T01:43:22.431300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:09.076123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:43.442351+0000","last_change":"2024-04-25T01:43:17.023058+0000","last_active":"2024-04-25T01:43:43.442351+0000","last_peered":"2024-04-25T01:43:43.442351+0000","last_clean":"2024-04-25T01:43:43.442351+0000","last_became_active":"2024-04-25T01:43:17.022730+0000","last_became_peered":"2024-04-25T01:43:17.022730+0000","last_unstale":"2024-04-25T01:43:43.442351+0000","last_undegraded":"2024-04-25T01:43:43.442351+0000","last_fullsized":"2024-04-25T01:43:43.442351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:12.279161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158616+0000","last_change":"2024-04-25T01:43:17.428832+0000","last_active":"2024-04-25T01:43:23.158616+0000","last_peered":"2024-04-25T01:43:23.158616+0000","last_clean":"2024-04-25T01:43:23.158616+0000","last_became_active":"2024-04-25T01:43:17.428493+0000","last_became_peered":"2024-04-25T01:43:17.428493+0000","last_unstale":"2024-04-25T01:43:23.158616+0000","last_undegraded":"2024-04-25T01:43:23.158616+0000","last_fullsized":"2024-04-25T01:43:23.158616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:47.224987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157502+0000","last_change":"2024-04-25T01:43:17.429227+0000","last_active":"2024-04-25T01:43:23.157502+0000","last_peered":"2024-04-25T01:43:23.157502+0000","last_clean":"2024-04-25T01:43:23.157502+0000","last_became_active":"2024-04-25T01:43:17.428693+0000","last_became_peered":"2024-04-25T01:43:17.428693+0000","last_unstale":"2024-04-25T01:43:23.157502+0000","last_undegraded":"2024-04-25T01:43:23.157502+0000","last_fullsized":"2024-04-25T01:43:23.157502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:58.511414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157309+0000","last_change":"2024-04-25T01:43:17.425880+0000","last_active":"2024-04-25T01:43:23.157309+0000","last_peered":"2024-04-25T01:43:23.157309+0000","last_clean":"2024-04-25T01:43:23.157309+0000","last_became_active":"2024-04-25T01:43:17.425539+0000","last_became_peered":"2024-04-25T01:43:17.425539+0000","last_unstale":"2024-04-25T01:43:23.157309+0000","last_undegraded":"2024-04-25T01:43:23.157309+0000","last_fullsized":"2024-04-25T01:43:23.157309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:40.909991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157272+0000","last_change":"2024-04-25T01:43:17.421214+0000","last_active":"2024-04-25T01:43:23.157272+0000","last_peered":"2024-04-25T01:43:23.157272+0000","last_clean":"2024-04-25T01:43:23.157272+0000","last_became_active":"2024-04-25T01:43:17.420872+0000","last_became_peered":"2024-04-25T01:43:17.420872+0000","last_unstale":"2024-04-25T01:43:23.157272+0000","last_undegraded":"2024-04-25T01:43:23.157272+0000","last_fullsized":"2024-04-25T01:43:23.157272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:48.066191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157056+0000","last_change":"2024-04-25T01:43:17.426011+0000","last_active":"2024-04-25T01:43:23.157056+0000","last_peered":"2024-04-25T01:43:23.157056+0000","last_clean":"2024-04-25T01:43:23.157056+0000","last_became_active":"2024-04-25T01:43:17.425629+0000","last_became_peered":"2024-04-25T01:43:17.425629+0000","last_unstale":"2024-04-25T01:43:23.157056+0000","last_undegraded":"2024-04-25T01:43:23.157056+0000","last_fullsized":"2024-04-25T01:43:23.157056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:12.606749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431170+0000","last_change":"2024-04-25T01:43:17.421374+0000","last_active":"2024-04-25T01:43:22.431170+0000","last_peered":"2024-04-25T01:43:22.431170+0000","last_clean":"2024-04-25T01:43:22.431170+0000","last_became_active":"2024-04-25T01:43:17.420864+0000","last_became_peered":"2024-04-25T01:43:17.420864+0000","last_unstale":"2024-04-25T01:43:22.431170+0000","last_undegraded":"2024-04-25T01:43:22.431170+0000","last_fullsized":"2024-04-25T01:43:22.431170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:17.517927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157716+0000","last_change":"2024-04-25T01:43:17.431131+0000","last_active":"2024-04-25T01:43:23.157716+0000","last_peered":"2024-04-25T01:43:23.157716+0000","last_clean":"2024-04-25T01:43:23.157716+0000","last_became_active":"2024-04-25T01:43:17.430809+0000","last_became_peered":"2024-04-25T01:43:17.430809+0000","last_unstale":"2024-04-25T01:43:23.157716+0000","last_undegraded":"2024-04-25T01:43:23.157716+0000","last_fullsized":"2024-04-25T01:43:23.157716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:48.261501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.158423+0000","last_change":"2024-04-25T01:43:17.419104+0000","last_active":"2024-04-25T01:43:23.158423+0000","last_peered":"2024-04-25T01:43:23.158423+0000","last_clean":"2024-04-25T01:43:23.158423+0000","last_became_active":"2024-04-25T01:43:17.418799+0000","last_became_peered":"2024-04-25T01:43:17.418799+0000","last_unstale":"2024-04-25T01:43:23.158423+0000","last_undegraded":"2024-04-25T01:43:23.158423+0000","last_fullsized":"2024-04-25T01:43:23.158423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:46.000847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.113092+0000","last_change":"2024-04-25T01:43:17.425801+0000","last_active":"2024-04-25T01:43:23.113092+0000","last_peered":"2024-04-25T01:43:23.113092+0000","last_clean":"2024-04-25T01:43:23.113092+0000","last_became_active":"2024-04-25T01:43:17.425133+0000","last_became_peered":"2024-04-25T01:43:17.425133+0000","last_unstale":"2024-04-25T01:43:23.113092+0000","last_undegraded":"2024-04-25T01:43:23.113092+0000","last_fullsized":"2024-04-25T01:43:23.113092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:54.229265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.431493+0000","last_change":"2024-04-25T01:43:17.423931+0000","last_active":"2024-04-25T01:43:22.431493+0000","last_peered":"2024-04-25T01:43:22.431493+0000","last_clean":"2024-04-25T01:43:22.431493+0000","last_became_active":"2024-04-25T01:43:17.423477+0000","last_became_peered":"2024-04-25T01:43:17.423477+0000","last_unstale":"2024-04-25T01:43:22.431493+0000","last_undegraded":"2024-04-25T01:43:22.431493+0000","last_fullsized":"2024-04-25T01:43:22.431493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:31.538936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.112801+0000","last_change":"2024-04-25T01:43:17.428469+0000","last_active":"2024-04-25T01:43:23.112801+0000","last_peered":"2024-04-25T01:43:23.112801+0000","last_clean":"2024-04-25T01:43:23.112801+0000","last_became_active":"2024-04-25T01:43:17.428121+0000","last_became_peered":"2024-04-25T01:43:17.428121+0000","last_unstale":"2024-04-25T01:43:23.112801+0000","last_undegraded":"2024-04-25T01:43:23.112801+0000","last_fullsized":"2024-04-25T01:43:23.112801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:27.563439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.430016+0000","last_change":"2024-04-25T01:43:17.433014+0000","last_active":"2024-04-25T01:43:22.430016+0000","last_peered":"2024-04-25T01:43:22.430016+0000","last_clean":"2024-04-25T01:43:22.430016+0000","last_became_active":"2024-04-25T01:43:17.432568+0000","last_became_peered":"2024-04-25T01:43:17.432568+0000","last_unstale":"2024-04-25T01:43:22.430016+0000","last_undegraded":"2024-04-25T01:43:22.430016+0000","last_fullsized":"2024-04-25T01:43:22.430016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:18.494922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:22.430921+0000","last_change":"2024-04-25T01:43:17.436079+0000","last_active":"2024-04-25T01:43:22.430921+0000","last_peered":"2024-04-25T01:43:22.430921+0000","last_clean":"2024-04-25T01:43:22.430921+0000","last_became_active":"2024-04-25T01:43:17.435735+0000","last_became_peered":"2024-04-25T01:43:17.435735+0000","last_unstale":"2024-04-25T01:43:22.430921+0000","last_undegraded":"2024-04-25T01:43:22.430921+0000","last_fullsized":"2024-04-25T01:43:22.430921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:59.593710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T01:43:23.157724+0000","last_change":"2024-04-25T01:43:17.424956+0000","last_active":"2024-04-25T01:43:23.157724+0000","last_peered":"2024-04-25T01:43:23.157724+0000","last_clean":"2024-04-25T01:43:23.157724+0000","last_became_active":"2024-04-25T01:43:17.424629+0000","last_became_peered":"2024-04-25T01:43:17.424629+0000","last_unstale":"2024-04-25T01:43:23.157724+0000","last_undegraded":"2024-04-25T01:43:23.157724+0000","last_fullsized":"2024-04-25T01:43:23.157724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:58.006952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245924,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9863,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":131,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":45883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":482901,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612631,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1076875,"data_compressed":50529,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776156,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":482291,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":482313,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1074932,"data_compressed":50529,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462956,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":482431,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":485551,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":481799,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":487523,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":481993,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":482800,"data_compressed":14683,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724593,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":1075288,"data_compressed":50529,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:43:46.132 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T01:43:46.133 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T01:43:46.133 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T01:43:46.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:45 smithi152 ceph-mon[32465]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:46.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:45 smithi152 ceph-mon[32465]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:46.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:45 smithi152 ceph-mon[32465]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:46.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:45 smithi152 ceph-mon[32465]: pgmap v304: 129 pgs: 129 active+clean; 586 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-25T01:43:46.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:45 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/33290992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:46.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:45 smithi152 ceph-mon[32465]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T01:43:46.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:45 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/489610415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:46.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:45 smithi196 ceph-mon[32978]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:46.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:45 smithi196 ceph-mon[32978]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:46.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:45 smithi196 ceph-mon[32978]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:46.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:45 smithi196 ceph-mon[32978]: pgmap v304: 129 pgs: 129 active+clean; 586 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-25T01:43:46.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:45 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/33290992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:43:46.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:45 smithi196 ceph-mon[32978]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T01:43:46.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:45 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/489610415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:46.405 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:43:47.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:46 smithi078 ceph-mon[29469]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:43:47.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:46 smithi152 ceph-mon[32465]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:43:47.298 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:43:46 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-b[62804]: 2024-04-25T01:43:46.961+0000 7faf6a3c9700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-25T01:43:47.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:47 smithi196 ceph-mon[32978]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:43:48.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.207+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1876638251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca08074c90 msgr2=0x7fca08075110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:48.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.207+0000 7fca0e2b5700 1 --2- 172.21.15.78:0/1876638251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca08074c90 0x7fca08075110 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc009990 tx=0x7fc9fc009ca0 comp rx=0 tx=0).stop 2024-04-25T01:43:48.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.207+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1876638251 shutdown_connections 2024-04-25T01:43:48.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.207+0000 7fca0e2b5700 1 --2- 172.21.15.78:0/1876638251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca0810e8a0 0x7fca08110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:48.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.207+0000 7fca0e2b5700 1 --2- 172.21.15.78:0/1876638251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08076260 0x7fca080746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:48.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.207+0000 7fca0e2b5700 1 --2- 172.21.15.78:0/1876638251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca08074c90 0x7fca08075110 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:48.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.207+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1876638251 >> 172.21.15.78:0/1876638251 conn(0x7fca080fd880 msgr2=0x7fca080ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:48.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.207+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1876638251 shutdown_connections 2024-04-25T01:43:48.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.207+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1876638251 wait complete. 2024-04-25T01:43:48.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:48 smithi078 ceph-mon[29469]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:43:48.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:48 smithi078 ceph-mon[29469]: pgmap v305: 129 pgs: 129 active+clean; 587 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.3 KiB/s wr, 14 op/s 2024-04-25T01:43:48.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.207+0000 7fca0e2b5700 1 Processor -- start 2024-04-25T01:43:48.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.207+0000 7fca0e2b5700 1 -- start start 2024-04-25T01:43:48.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca0e2b5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08074c90 0x7fca08106480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:48.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca0e2b5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca08076260 0x7fca081069c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:48.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca0e2b5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca0810e8a0 0x7fca0810da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca0e2b5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fca081145a0 con 0x7fca08076260 2024-04-25T01:43:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca0e2b5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fca08114420 con 0x7fca08074c90 2024-04-25T01:43:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca0e2b5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca08114720 con 0x7fca0810e8a0 2024-04-25T01:43:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca077fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca08076260 0x7fca081069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca077fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca08076260 0x7fca081069c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58652/0 (socket says 172.21.15.78:58652) 2024-04-25T01:43:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca077fe700 1 -- 172.21.15.78:0/1163378950 learned_addr learned my addr 172.21.15.78:0/1163378950 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca0c852700 1 --2- 172.21.15.78:0/1163378950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca0810e8a0 0x7fca0810da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca07fff700 1 --2- 172.21.15.78:0/1163378950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08074c90 0x7fca08106480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca077fe700 1 -- 172.21.15.78:0/1163378950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca0810e8a0 msgr2=0x7fca0810da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca077fe700 1 --2- 172.21.15.78:0/1163378950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca0810e8a0 0x7fca0810da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca077fe700 1 -- 172.21.15.78:0/1163378950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08074c90 msgr2=0x7fca08106480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca077fe700 1 --2- 172.21.15.78:0/1163378950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08074c90 0x7fca08106480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:48.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.208+0000 7fca077fe700 1 -- 172.21.15.78:0/1163378950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0810e140 con 0x7fca08076260 2024-04-25T01:43:48.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.209+0000 7fca077fe700 1 --2- 172.21.15.78:0/1163378950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca08076260 0x7fca081069c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc000c00 tx=0x7fc9fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:48.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.209+0000 7fca057fa700 1 -- 172.21.15.78:0/1163378950 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc027070 con 0x7fca08076260 2024-04-25T01:43:48.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.209+0000 7fca057fa700 1 -- 172.21.15.78:0/1163378950 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9fc004210 con 0x7fca08076260 2024-04-25T01:43:48.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.209+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1163378950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0810e3d0 con 0x7fca08076260 2024-04-25T01:43:48.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.209+0000 7fca057fa700 1 -- 172.21.15.78:0/1163378950 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc00f460 con 0x7fca08076260 2024-04-25T01:43:48.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.209+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1163378950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca081af970 con 0x7fca08076260 2024-04-25T01:43:48.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.210+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1163378950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f4005360 con 0x7fca08076260 2024-04-25T01:43:48.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.211+0000 7fca057fa700 1 -- 172.21.15.78:0/1163378950 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc9fc02a070 con 0x7fca08076260 2024-04-25T01:43:48.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.212+0000 7fca057fa700 1 --2- 172.21.15.78:0/1163378950 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc9f00a2de0 0x7fc9f00a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:48.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.212+0000 7fca07fff700 1 --2- 172.21.15.78:0/1163378950 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc9f00a2de0 0x7fc9f00a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:48.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.212+0000 7fca057fa700 1 -- 172.21.15.78:0/1163378950 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc9fc0d0230 con 0x7fca08076260 2024-04-25T01:43:48.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.212+0000 7fca07fff700 1 --2- 172.21.15.78:0/1163378950 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc9f00a2de0 0x7fc9f00a52a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fca08076640 tx=0x7fc9f8009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:48.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.216+0000 7fca057fa700 1 -- 172.21.15.78:0/1163378950 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc9fc05e740 con 0x7fca08076260 2024-04-25T01:43:48.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:48 smithi152 ceph-mon[32465]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:43:48.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:48 smithi152 ceph-mon[32465]: pgmap v305: 129 pgs: 129 active+clean; 587 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.3 KiB/s wr, 14 op/s 2024-04-25T01:43:48.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:48 smithi196 ceph-mon[32978]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:43:48.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:48 smithi196 ceph-mon[32978]: pgmap v305: 129 pgs: 129 active+clean; 587 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.3 KiB/s wr, 14 op/s 2024-04-25T01:43:48.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.491+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1163378950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc9f4005650 con 0x7fca08076260 2024-04-25T01:43:48.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.493+0000 7fca057fa700 1 -- 172.21.15.78:0/1163378950 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fc9fc013070 con 0x7fca08076260 2024-04-25T01:43:48.484 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-04-25T01:43:48.485 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBUtSlmsy9mHRAA/qOmHbyWi9sl2BdXt0NpqA== 2024-04-25T01:43:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.495+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1163378950 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc9f00a2de0 msgr2=0x7fc9f00a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.495+0000 7fca0e2b5700 1 --2- 172.21.15.78:0/1163378950 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc9f00a2de0 0x7fc9f00a52a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fca08076640 tx=0x7fc9f8009190 comp rx=0 tx=0).stop 2024-04-25T01:43:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.495+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1163378950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca08076260 msgr2=0x7fca081069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.495+0000 7fca0e2b5700 1 --2- 172.21.15.78:0/1163378950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca08076260 0x7fca081069c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc000c00 tx=0x7fc9fc0047b0 comp rx=0 tx=0).stop 2024-04-25T01:43:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.496+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1163378950 shutdown_connections 2024-04-25T01:43:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.496+0000 7fca0e2b5700 1 --2- 172.21.15.78:0/1163378950 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc9f00a2de0 0x7fc9f00a52a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.496+0000 7fca0e2b5700 1 --2- 172.21.15.78:0/1163378950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca0810e8a0 0x7fca0810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.496+0000 7fca0e2b5700 1 --2- 172.21.15.78:0/1163378950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08074c90 0x7fca08106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.496+0000 7fca0e2b5700 1 --2- 172.21.15.78:0/1163378950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca08076260 0x7fca081069c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.496+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1163378950 >> 172.21.15.78:0/1163378950 conn(0x7fca080fd880 msgr2=0x7fca08101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:48.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.496+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1163378950 shutdown_connections 2024-04-25T01:43:48.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:48.496+0000 7fca0e2b5700 1 -- 172.21.15.78:0/1163378950 wait complete. 2024-04-25T01:43:49.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:49 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1163378950' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T01:43:49.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:49 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1163378950' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T01:43:49.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:49 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1163378950' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T01:43:49.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:49 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1163378950' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T01:43:49.321 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:43:49.321 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T01:43:49.321 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T01:43:49.357 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T01:43:49.357 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T01:43:49.357 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T01:43:49.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:49 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1163378950' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T01:43:49.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:49 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1163378950' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T01:43:49.644 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:43:50.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:50 smithi078 ceph-mon[29469]: pgmap v306: 129 pgs: 129 active+clean; 629 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-04-25T01:43:50.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:50 smithi152 ceph-mon[32465]: pgmap v306: 129 pgs: 129 active+clean; 629 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-04-25T01:43:50.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:50 smithi196 ceph-mon[32978]: pgmap v306: 129 pgs: 129 active+clean; 629 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-04-25T01:43:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.373+0000 7fab947f0700 1 -- 172.21.15.78:0/3911196975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c102e90 msgr2=0x7fab8c106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.373+0000 7fab947f0700 1 --2- 172.21.15.78:0/3911196975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c102e90 0x7fab8c106e90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fab88009990 tx=0x7fab88009ca0 comp rx=0 tx=0).stop 2024-04-25T01:43:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.373+0000 7fab947f0700 1 -- 172.21.15.78:0/3911196975 shutdown_connections 2024-04-25T01:43:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.373+0000 7fab947f0700 1 --2- 172.21.15.78:0/3911196975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c107570 0x7fab8c10f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.373+0000 7fab947f0700 1 --2- 172.21.15.78:0/3911196975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c1024e0 0x7fab8c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.373+0000 7fab947f0700 1 --2- 172.21.15.78:0/3911196975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c102e90 0x7fab8c106e90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.373+0000 7fab947f0700 1 -- 172.21.15.78:0/3911196975 >> 172.21.15.78:0/3911196975 conn(0x7fab8c079390 msgr2=0x7fab8c0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.374+0000 7fab947f0700 1 -- 172.21.15.78:0/3911196975 shutdown_connections 2024-04-25T01:43:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.374+0000 7fab947f0700 1 -- 172.21.15.78:0/3911196975 wait complete. 2024-04-25T01:43:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.374+0000 7fab947f0700 1 Processor -- start 2024-04-25T01:43:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.374+0000 7fab947f0700 1 -- start start 2024-04-25T01:43:51.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.374+0000 7fab947f0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c1024e0 0x7fab8c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.374+0000 7fab947f0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102e90 0x7fab8c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.374+0000 7fab947f0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c107570 0x7fab8c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.374+0000 7fab947f0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c112590 con 0x7fab8c1024e0 2024-04-25T01:43:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.374+0000 7fab947f0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c112410 con 0x7fab8c107570 2024-04-25T01:43:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab947f0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c112710 con 0x7fab8c102e90 2024-04-25T01:43:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab9258c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c1024e0 0x7fab8c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab9258c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c1024e0 0x7fab8c19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39194/0 (socket says 172.21.15.78:39194) 2024-04-25T01:43:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab9258c700 1 -- 172.21.15.78:0/898231724 learned_addr learned my addr 172.21.15.78:0/898231724 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab92d8d700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c107570 0x7fab8c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab91d8b700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102e90 0x7fab8c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab9258c700 1 -- 172.21.15.78:0/898231724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102e90 msgr2=0x7fab8c19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab9258c700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102e90 0x7fab8c19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:51.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab9258c700 1 -- 172.21.15.78:0/898231724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c107570 msgr2=0x7fab8c198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab9258c700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c107570 0x7fab8c198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab9258c700 1 -- 172.21.15.78:0/898231724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c199010 con 0x7fab8c1024e0 2024-04-25T01:43:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab91d8b700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102e90 0x7fab8c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:43:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab9258c700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c1024e0 0x7fab8c19e6b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00b530 tx=0x7fab7c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab837fe700 1 -- 172.21.15.78:0/898231724 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c015020 con 0x7fab8c1024e0 2024-04-25T01:43:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab947f0700 1 -- 172.21.15.78:0/898231724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c199300 con 0x7fab8c1024e0 2024-04-25T01:43:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab837fe700 1 -- 172.21.15.78:0/898231724 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab7c011d10 con 0x7fab8c1024e0 2024-04-25T01:43:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab837fe700 1 -- 172.21.15.78:0/898231724 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c010e50 con 0x7fab8c1024e0 2024-04-25T01:43:51.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.375+0000 7fab947f0700 1 -- 172.21.15.78:0/898231724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c1a5510 con 0x7fab8c1024e0 2024-04-25T01:43:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.377+0000 7fab947f0700 1 -- 172.21.15.78:0/898231724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8c103970 con 0x7fab8c1024e0 2024-04-25T01:43:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.378+0000 7fab837fe700 1 -- 172.21.15.78:0/898231724 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fab7c018020 con 0x7fab8c1024e0 2024-04-25T01:43:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.378+0000 7fab837fe700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fab780a2e50 0x7fab780a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.378+0000 7fab91d8b700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fab780a2e50 0x7fab780a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.378+0000 7fab837fe700 1 -- 172.21.15.78:0/898231724 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fab7c0ceb70 con 0x7fab8c1024e0 2024-04-25T01:43:51.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.379+0000 7fab91d8b700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fab780a2e50 0x7fab780a5310 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fab88000c00 tx=0x7fab88005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:51.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.383+0000 7fab837fe700 1 -- 172.21.15.78:0/898231724 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fab7c01f020 con 0x7fab8c1024e0 2024-04-25T01:43:51.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.639+0000 7fab947f0700 1 -- 172.21.15.78:0/898231724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fab8c02db30 con 0x7fab8c1024e0 2024-04-25T01:43:51.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.644+0000 7fab837fe700 1 -- 172.21.15.78:0/898231724 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263201 (secure 0 0 0) 0x7fab7c09b840 con 0x7fab8c1024e0 2024-04-25T01:43:51.638 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.647+0000 7fab947f0700 1 -- 172.21.15.78:0/898231724 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fab780a2e50 msgr2=0x7fab780a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.647+0000 7fab947f0700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fab780a2e50 0x7fab780a5310 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fab88000c00 tx=0x7fab88005fb0 comp rx=0 tx=0).stop 2024-04-25T01:43:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.647+0000 7fab947f0700 1 -- 172.21.15.78:0/898231724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c1024e0 msgr2=0x7fab8c19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.647+0000 7fab947f0700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c1024e0 0x7fab8c19e6b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00b530 tx=0x7fab7c00b8f0 comp rx=0 tx=0).stop 2024-04-25T01:43:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.647+0000 7fab947f0700 1 -- 172.21.15.78:0/898231724 shutdown_connections 2024-04-25T01:43:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.647+0000 7fab947f0700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fab780a2e50 0x7fab780a5310 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.647+0000 7fab947f0700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c107570 0x7fab8c198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.647+0000 7fab947f0700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c1024e0 0x7fab8c19e6b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.647+0000 7fab947f0700 1 --2- 172.21.15.78:0/898231724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102e90 0x7fab8c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.647+0000 7fab947f0700 1 -- 172.21.15.78:0/898231724 >> 172.21.15.78:0/898231724 conn(0x7fab8c079390 msgr2=0x7fab8c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.647+0000 7fab947f0700 1 -- 172.21.15.78:0/898231724 shutdown_connections 2024-04-25T01:43:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:51.647+0000 7fab947f0700 1 -- 172.21.15.78:0/898231724 wait complete. 2024-04-25T01:43:52.255 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:52 smithi078 ceph-mon[29469]: pgmap v307: 129 pgs: 129 active+clean; 629 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.5 KiB/s wr, 22 op/s 2024-04-25T01:43:52.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:52 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/898231724' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T01:43:52.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:52 smithi152 ceph-mon[32465]: pgmap v307: 129 pgs: 129 active+clean; 629 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.5 KiB/s wr, 22 op/s 2024-04-25T01:43:52.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:52 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/898231724' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T01:43:52.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:52 smithi196 ceph-mon[32978]: pgmap v307: 129 pgs: 129 active+clean; 629 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.5 KiB/s wr, 22 op/s 2024-04-25T01:43:52.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:52 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/898231724' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T01:43:52.390 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":2190983290},{"type":"v1","addr":"172.21.15.78:6801","nonce":2190983290}]},"active_addr":"172.21.15.78:6801/2190983290","active_change":"2024-04-25T01:36:16.812655+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.78:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":1827952621}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2208238897}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2232805550}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":3723981482}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2624647573}]}]} 2024-04-25T01:43:52.395 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T01:43:52.395 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T01:43:52.395 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T01:43:52.635 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:43:54.126 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:54 smithi078 ceph-mon[29469]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 KiB/s wr, 25 op/s 2024-04-25T01:43:54.297 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:54 smithi152 ceph-mon[32465]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 KiB/s wr, 25 op/s 2024-04-25T01:43:54.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.351+0000 7f365b760700 1 -- 172.21.15.78:0/3222031687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36541082b0 msgr2=0x7f3654108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:54.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.351+0000 7f365b760700 1 --2- 172.21.15.78:0/3222031687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36541082b0 0x7f3654108690 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3644009990 tx=0x7f3644009ca0 comp rx=0 tx=0).stop 2024-04-25T01:43:54.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.351+0000 7f365b760700 1 -- 172.21.15.78:0/3222031687 shutdown_connections 2024-04-25T01:43:54.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.351+0000 7f365b760700 1 --2- 172.21.15.78:0/3222031687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365406b320 0x7f365410f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:54.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.351+0000 7f365b760700 1 --2- 172.21.15.78:0/3222031687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36541082b0 0x7f3654108690 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:54.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.351+0000 7f365b760700 1 --2- 172.21.15.78:0/3222031687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365406a980 0x7f365406ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:54.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.351+0000 7f365b760700 1 -- 172.21.15.78:0/3222031687 >> 172.21.15.78:0/3222031687 conn(0x7f3654078100 msgr2=0x7f3654078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:54.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.351+0000 7f365b760700 1 -- 172.21.15.78:0/3222031687 shutdown_connections 2024-04-25T01:43:54.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.351+0000 7f365b760700 1 -- 172.21.15.78:0/3222031687 wait complete. 2024-04-25T01:43:54.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.352+0000 7f365b760700 1 Processor -- start 2024-04-25T01:43:54.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.352+0000 7f365b760700 1 -- start start 2024-04-25T01:43:54.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.352+0000 7f365b760700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365406a980 0x7f3654101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:54.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.352+0000 7f365b760700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365406b320 0x7f3654101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:54.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.352+0000 7f365b760700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36541082b0 0x7f3654101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:54.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.352+0000 7f365b760700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3654112510 con 0x7f36541082b0 2024-04-25T01:43:54.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.352+0000 7f365b760700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3654112390 con 0x7f365406a980 2024-04-25T01:43:54.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.352+0000 7f365b760700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3654112690 con 0x7f365406b320 2024-04-25T01:43:54.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.352+0000 7f3659cfd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36541082b0 0x7f3654101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:54.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.352+0000 7f3659cfd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36541082b0 0x7f3654101cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39216/0 (socket says 172.21.15.78:39216) 2024-04-25T01:43:54.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.352+0000 7f3659cfd700 1 -- 172.21.15.78:0/2446326178 learned_addr learned my addr 172.21.15.78:0/2446326178 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:54.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.353+0000 7f3658cfb700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365406b320 0x7f3654101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:54.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.353+0000 7f36594fc700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365406a980 0x7f3654101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:54.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.353+0000 7f3659cfd700 1 -- 172.21.15.78:0/2446326178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365406b320 msgr2=0x7f3654101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:54.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.353+0000 7f3659cfd700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365406b320 0x7f3654101770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:54.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.353+0000 7f3659cfd700 1 -- 172.21.15.78:0/2446326178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365406a980 msgr2=0x7f3654101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:54.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.353+0000 7f3659cfd700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365406a980 0x7f3654101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.353+0000 7f3659cfd700 1 -- 172.21.15.78:0/2446326178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36541a6f30 con 0x7f36541082b0 2024-04-25T01:43:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.353+0000 7f36594fc700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365406a980 0x7f3654101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:43:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.353+0000 7f3659cfd700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36541082b0 0x7f3654101cb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f364c007ee0 tx=0x7f364c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.353+0000 7f364a7fc700 1 -- 172.21.15.78:0/2446326178 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c00dbd0 con 0x7f36541082b0 2024-04-25T01:43:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.353+0000 7f365b760700 1 -- 172.21.15.78:0/2446326178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36541a7220 con 0x7f36541082b0 2024-04-25T01:43:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.353+0000 7f365b760700 1 -- 172.21.15.78:0/2446326178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36541a77b0 con 0x7f36541082b0 2024-04-25T01:43:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.353+0000 7f364a7fc700 1 -- 172.21.15.78:0/2446326178 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f364c00dd70 con 0x7f36541082b0 2024-04-25T01:43:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.353+0000 7f364a7fc700 1 -- 172.21.15.78:0/2446326178 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c0133f0 con 0x7f36541082b0 2024-04-25T01:43:54.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.354+0000 7f365b760700 1 -- 172.21.15.78:0/2446326178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3638005330 con 0x7f36541082b0 2024-04-25T01:43:54.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.355+0000 7f364a7fc700 1 -- 172.21.15.78:0/2446326178 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f364c01a020 con 0x7f36541082b0 2024-04-25T01:43:54.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.356+0000 7f364a7fc700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36400a2cb0 0x7f36400a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:54.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.356+0000 7f36594fc700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36400a2cb0 0x7f36400a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:54.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.356+0000 7f364a7fc700 1 -- 172.21.15.78:0/2446326178 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f364c0cf020 con 0x7f36541082b0 2024-04-25T01:43:54.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.356+0000 7f36594fc700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36400a2cb0 0x7f36400a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f364400b2e0 tx=0x7f3644005e80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:54.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.359+0000 7f364a7fc700 1 -- 172.21.15.78:0/2446326178 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f364c010070 con 0x7f36541082b0 2024-04-25T01:43:54.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:54 smithi196 ceph-mon[32978]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 KiB/s wr, 25 op/s 2024-04-25T01:43:54.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.559+0000 7f365b760700 1 -- 172.21.15.78:0/2446326178 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36380058f0 con 0x7f36541082b0 2024-04-25T01:43:54.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.560+0000 7f364a7fc700 1 -- 172.21.15.78:0/2446326178 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24151 (secure 0 0 0) 0x7f364c0192c0 con 0x7f36541082b0 2024-04-25T01:43:54.552 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:54.552 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:43:21.781660+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:43:54.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.563+0000 7f365b760700 1 -- 172.21.15.78:0/2446326178 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36400a2cb0 msgr2=0x7f36400a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:54.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.563+0000 7f365b760700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36400a2cb0 0x7f36400a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f364400b2e0 tx=0x7f3644005e80 comp rx=0 tx=0).stop 2024-04-25T01:43:54.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.563+0000 7f365b760700 1 -- 172.21.15.78:0/2446326178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36541082b0 msgr2=0x7f3654101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:54.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.563+0000 7f365b760700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36541082b0 0x7f3654101cb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f364c007ee0 tx=0x7f364c00d3b0 comp rx=0 tx=0).stop 2024-04-25T01:43:54.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.563+0000 7f365b760700 1 -- 172.21.15.78:0/2446326178 shutdown_connections 2024-04-25T01:43:54.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.563+0000 7f365b760700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36400a2cb0 0x7f36400a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:54.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.563+0000 7f365b760700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36541082b0 0x7f3654101cb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:54.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.563+0000 7f365b760700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365406a980 0x7f3654101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:54.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.563+0000 7f365b760700 1 --2- 172.21.15.78:0/2446326178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365406b320 0x7f3654101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:54.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.563+0000 7f365b760700 1 -- 172.21.15.78:0/2446326178 >> 172.21.15.78:0/2446326178 conn(0x7f3654078100 msgr2=0x7f36540fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:54.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.563+0000 7f365b760700 1 -- 172.21.15.78:0/2446326178 shutdown_connections 2024-04-25T01:43:54.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:54.563+0000 7f365b760700 1 -- 172.21.15.78:0/2446326178 wait complete. 2024-04-25T01:43:55.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:55 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2446326178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:55.355 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:55 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2446326178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:55.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:55 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2446326178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:55.389 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T01:43:55.389 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T01:43:55.632 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:43:56.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:56 smithi152 ceph-mon[32465]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2024-04-25T01:43:56.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:56 smithi196 ceph-mon[32978]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2024-04-25T01:43:56.454 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:56 smithi078 ceph-mon[29469]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2024-04-25T01:43:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.267+0000 7f2eebcf0700 1 -- 172.21.15.78:0/926810514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee41075a0 msgr2=0x7f2ee410f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.267+0000 7f2eebcf0700 1 --2- 172.21.15.78:0/926810514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee41075a0 0x7f2ee410f170 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00b0a0 tx=0x7f2edc00b3b0 comp rx=0 tx=0).stop 2024-04-25T01:43:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.267+0000 7f2eebcf0700 1 -- 172.21.15.78:0/926810514 shutdown_connections 2024-04-25T01:43:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.267+0000 7f2eebcf0700 1 --2- 172.21.15.78:0/926810514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ee4102540 0x7f2ee4102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.267+0000 7f2eebcf0700 1 --2- 172.21.15.78:0/926810514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee41075a0 0x7f2ee410f170 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.267+0000 7f2eebcf0700 1 --2- 172.21.15.78:0/926810514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ee4102ef0 0x7f2ee4106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.267+0000 7f2eebcf0700 1 -- 172.21.15.78:0/926810514 >> 172.21.15.78:0/926810514 conn(0x7f2ee4079310 msgr2=0x7f2ee4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.267+0000 7f2eebcf0700 1 -- 172.21.15.78:0/926810514 shutdown_connections 2024-04-25T01:43:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.267+0000 7f2eebcf0700 1 -- 172.21.15.78:0/926810514 wait complete. 2024-04-25T01:43:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2eebcf0700 1 Processor -- start 2024-04-25T01:43:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2eebcf0700 1 -- start start 2024-04-25T01:43:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2eebcf0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102540 0x7f2ee4074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2eebcf0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ee4102ef0 0x7f2ee4074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2eebcf0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ee41075a0 0x7f2ee4074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2eebcf0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4112660 con 0x7f2ee4102540 2024-04-25T01:43:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2eebcf0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41124e0 con 0x7f2ee4102ef0 2024-04-25T01:43:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2eebcf0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41127e0 con 0x7f2ee41075a0 2024-04-25T01:43:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2ee9a8c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102540 0x7f2ee4074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2ee9a8c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102540 0x7f2ee4074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39240/0 (socket says 172.21.15.78:39240) 2024-04-25T01:43:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2ee9a8c700 1 -- 172.21.15.78:0/3721709014 learned_addr learned my addr 172.21.15.78:0/3721709014 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:43:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2eea28d700 1 --2- 172.21.15.78:0/3721709014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ee41075a0 0x7f2ee4074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2ee928b700 1 --2- 172.21.15.78:0/3721709014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ee4102ef0 0x7f2ee4074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2ee9a8c700 1 -- 172.21.15.78:0/3721709014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ee41075a0 msgr2=0x7f2ee4074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2ee9a8c700 1 --2- 172.21.15.78:0/3721709014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ee41075a0 0x7f2ee4074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2ee9a8c700 1 -- 172.21.15.78:0/3721709014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ee4102ef0 msgr2=0x7f2ee4074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2ee9a8c700 1 --2- 172.21.15.78:0/3721709014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ee4102ef0 0x7f2ee4074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.268+0000 7f2ee9a8c700 1 -- 172.21.15.78:0/3721709014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee41a6fc0 con 0x7f2ee4102540 2024-04-25T01:43:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.269+0000 7f2ee9a8c700 1 --2- 172.21.15.78:0/3721709014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102540 0x7f2ee4074530 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400b530 tx=0x7f2ed400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.269+0000 7f2edaffd700 1 -- 172.21.15.78:0/3721709014 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4015020 con 0x7f2ee4102540 2024-04-25T01:43:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.269+0000 7f2eebcf0700 1 -- 172.21.15.78:0/3721709014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee41a72b0 con 0x7f2ee4102540 2024-04-25T01:43:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.269+0000 7f2edaffd700 1 -- 172.21.15.78:0/3721709014 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed4011d10 con 0x7f2ee4102540 2024-04-25T01:43:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.269+0000 7f2edaffd700 1 -- 172.21.15.78:0/3721709014 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400f2e0 con 0x7f2ee4102540 2024-04-25T01:43:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.269+0000 7f2eebcf0700 1 -- 172.21.15.78:0/3721709014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee41a7840 con 0x7f2ee4102540 2024-04-25T01:43:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.270+0000 7f2eebcf0700 1 -- 172.21.15.78:0/3721709014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee4068f20 con 0x7f2ee4102540 2024-04-25T01:43:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.271+0000 7f2edaffd700 1 -- 172.21.15.78:0/3721709014 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2ed4018020 con 0x7f2ee4102540 2024-04-25T01:43:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.271+0000 7f2edaffd700 1 --2- 172.21.15.78:0/3721709014 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2ed00a2cb0 0x7f2ed00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.272+0000 7f2ee928b700 1 --2- 172.21.15.78:0/3721709014 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2ed00a2cb0 0x7f2ed00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.272+0000 7f2edaffd700 1 -- 172.21.15.78:0/3721709014 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2ed40cef60 con 0x7f2ee4102540 2024-04-25T01:43:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.272+0000 7f2ee928b700 1 --2- 172.21.15.78:0/3721709014 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2ed00a2cb0 0x7f2ed00a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4079d00 tx=0x7f2ee0009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.275+0000 7f2edaffd700 1 -- 172.21.15.78:0/3721709014 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2ed401f020 con 0x7f2ee4102540 2024-04-25T01:43:57.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.477+0000 7f2eebcf0700 1 -- 172.21.15.78:0/3721709014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ee402db30 con 0x7f2ee4102540 2024-04-25T01:43:57.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.479+0000 7f2edaffd700 1 -- 172.21.15.78:0/3721709014 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24151 (secure 0 0 0) 0x7f2ed409bc30 con 0x7f2ee4102540 2024-04-25T01:43:57.471 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:43:57.471 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:43:21.781660+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:43:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.481+0000 7f2eebcf0700 1 -- 172.21.15.78:0/3721709014 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2ed00a2cb0 msgr2=0x7f2ed00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.481+0000 7f2eebcf0700 1 --2- 172.21.15.78:0/3721709014 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2ed00a2cb0 0x7f2ed00a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4079d00 tx=0x7f2ee0009190 comp rx=0 tx=0).stop 2024-04-25T01:43:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.481+0000 7f2eebcf0700 1 -- 172.21.15.78:0/3721709014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102540 msgr2=0x7f2ee4074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.481+0000 7f2eebcf0700 1 --2- 172.21.15.78:0/3721709014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102540 0x7f2ee4074530 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400b530 tx=0x7f2ed400b840 comp rx=0 tx=0).stop 2024-04-25T01:43:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.481+0000 7f2eebcf0700 1 -- 172.21.15.78:0/3721709014 shutdown_connections 2024-04-25T01:43:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.481+0000 7f2eebcf0700 1 --2- 172.21.15.78:0/3721709014 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2ed00a2cb0 0x7f2ed00a5170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.481+0000 7f2eebcf0700 1 --2- 172.21.15.78:0/3721709014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ee41075a0 0x7f2ee4074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.481+0000 7f2eebcf0700 1 --2- 172.21.15.78:0/3721709014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102540 0x7f2ee4074530 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.481+0000 7f2eebcf0700 1 --2- 172.21.15.78:0/3721709014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ee4102ef0 0x7f2ee4074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.481+0000 7f2eebcf0700 1 -- 172.21.15.78:0/3721709014 >> 172.21.15.78:0/3721709014 conn(0x7f2ee4079310 msgr2=0x7f2ee41002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.481+0000 7f2eebcf0700 1 -- 172.21.15.78:0/3721709014 shutdown_connections 2024-04-25T01:43:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:43:57.481+0000 7f2eebcf0700 1 -- 172.21.15.78:0/3721709014 wait complete. 2024-04-25T01:43:58.198 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:58 smithi078 ceph-mon[29469]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 KiB/s wr, 19 op/s 2024-04-25T01:43:58.198 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:43:58 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3721709014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:58.228 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T01:43:58.229 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T01:43:58.229 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T01:43:58.229 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T01:43:58.230 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T01:43:58.230 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T01:43:58.231 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T01:43:58.231 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T01:43:58.231 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T01:43:58.232 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T01:43:58.232 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T01:43:58.233 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T01:43:58.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:58 smithi152 ceph-mon[32465]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 KiB/s wr, 19 op/s 2024-04-25T01:43:58.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:43:58 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3721709014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:58.365 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:58 smithi196 ceph-mon[32978]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 KiB/s wr, 19 op/s 2024-04-25T01:43:58.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:43:58 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3721709014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:43:58.854 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:43:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:43:58.872 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:00.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:00 smithi078 ceph-mon[29469]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 15 op/s 2024-04-25T01:44:00.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:00 smithi152 ceph-mon[32465]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 15 op/s 2024-04-25T01:44:00.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:00 smithi196 ceph-mon[32978]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 15 op/s 2024-04-25T01:44:00.510 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:00.513 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:00.513 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:00.513 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:00.519 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:00.659 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:00.764 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:00.765 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:02.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:02 smithi078 ceph-mon[29469]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-04-25T01:44:02.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:02 smithi152 ceph-mon[32465]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-04-25T01:44:02.358 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:02 smithi196 ceph-mon[32978]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-04-25T01:44:02.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.835+0000 7f587b8f5700 1 -- 172.21.15.78:0/587652164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5874100820 msgr2=0x7f5874100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:02.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.835+0000 7f587b8f5700 1 --2- 172.21.15.78:0/587652164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5874100820 0x7f5874100c80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5870009990 tx=0x7f5870009ca0 comp rx=0 tx=0).stop 2024-04-25T01:44:02.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.835+0000 7f587b8f5700 1 -- 172.21.15.78:0/587652164 shutdown_connections 2024-04-25T01:44:02.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.835+0000 7f587b8f5700 1 --2- 172.21.15.78:0/587652164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58741011c0 0x7f587410f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:02.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.835+0000 7f587b8f5700 1 --2- 172.21.15.78:0/587652164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58741082c0 0x7f58741086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:02.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.835+0000 7f587b8f5700 1 --2- 172.21.15.78:0/587652164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5874100820 0x7f5874100c80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:02.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.835+0000 7f587b8f5700 1 -- 172.21.15.78:0/587652164 >> 172.21.15.78:0/587652164 conn(0x7f58740777a0 msgr2=0x7f58740feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:02.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.835+0000 7f587b8f5700 1 -- 172.21.15.78:0/587652164 shutdown_connections 2024-04-25T01:44:02.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.835+0000 7f587b8f5700 1 -- 172.21.15.78:0/587652164 wait complete. 2024-04-25T01:44:02.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f587b8f5700 1 Processor -- start 2024-04-25T01:44:02.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f587b8f5700 1 -- start start 2024-04-25T01:44:02.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f587b8f5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5874100820 0x7f587419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:02.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f587b8f5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58741011c0 0x7f587419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:02.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f587b8f5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58741082c0 0x7f5874198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:02.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f587b8f5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58741125a0 con 0x7f58741082c0 2024-04-25T01:44:02.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f587b8f5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5874112420 con 0x7f5874100820 2024-04-25T01:44:02.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f587b8f5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5874112720 con 0x7f58741011c0 2024-04-25T01:44:02.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f5879e92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58741082c0 0x7f5874198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:02.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f5879e92700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58741082c0 0x7f5874198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42666/0 (socket says 172.21.15.78:42666) 2024-04-25T01:44:02.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f5879e92700 1 -- 172.21.15.78:0/2066358566 learned_addr learned my addr 172.21.15.78:0/2066358566 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:02.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f5878e90700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58741011c0 0x7f587419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:02.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f5879691700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5874100820 0x7f587419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:02.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f5879e92700 1 -- 172.21.15.78:0/2066358566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58741011c0 msgr2=0x7f587419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:02.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f5879e92700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58741011c0 0x7f587419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:02.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f5879e92700 1 -- 172.21.15.78:0/2066358566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5874100820 msgr2=0x7f587419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f5879e92700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5874100820 0x7f587419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f5879e92700 1 -- 172.21.15.78:0/2066358566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5874199020 con 0x7f58741082c0 2024-04-25T01:44:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f5879691700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5874100820 0x7f587419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:44:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.836+0000 7f5879e92700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58741082c0 0x7f5874198840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f586c007ee0 tx=0x7f586c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.837+0000 7f586a7fc700 1 -- 172.21.15.78:0/2066358566 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c00dbd0 con 0x7f58741082c0 2024-04-25T01:44:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.837+0000 7f586a7fc700 1 -- 172.21.15.78:0/2066358566 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f586c00dd70 con 0x7f58741082c0 2024-04-25T01:44:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.837+0000 7f587b8f5700 1 -- 172.21.15.78:0/2066358566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5874199310 con 0x7f58741082c0 2024-04-25T01:44:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.837+0000 7f586a7fc700 1 -- 172.21.15.78:0/2066358566 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c005a60 con 0x7f58741082c0 2024-04-25T01:44:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.837+0000 7f587b8f5700 1 -- 172.21.15.78:0/2066358566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58741a5520 con 0x7f58741082c0 2024-04-25T01:44:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.837+0000 7f587b8f5700 1 -- 172.21.15.78:0/2066358566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58740629c0 con 0x7f58741082c0 2024-04-25T01:44:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.839+0000 7f586a7fc700 1 -- 172.21.15.78:0/2066358566 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f586c01a020 con 0x7f58741082c0 2024-04-25T01:44:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.839+0000 7f586a7fc700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f58600a2cb0 0x7f58600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.839+0000 7f5879691700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f58600a2cb0 0x7f58600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.839+0000 7f586a7fc700 1 -- 172.21.15.78:0/2066358566 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f586c0cf220 con 0x7f58741082c0 2024-04-25T01:44:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.839+0000 7f586a7fc700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253] conn(0x7f58600ad970 0x7f58600afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.839+0000 7f586a7fc700 1 -- 172.21.15.78:0/2066358566 --> [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f586c01de50 con 0x7f58600ad970 2024-04-25T01:44:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.839+0000 7f586a7fc700 1 -- 172.21.15.78:0/2066358566 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f586c0cf670 con 0x7f58741082c0 2024-04-25T01:44:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.840+0000 7f5879691700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f58600a2cb0 0x7f58600a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f58640099d0 tx=0x7f5864008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.840+0000 7f5878e90700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253] conn(0x7f58600ad970 0x7f58600afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.840+0000 7f5878e90700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253] conn(0x7f58600ad970 0x7f58600afdb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.841+0000 7f586a7fc700 1 -- 172.21.15.78:0/2066358566 <== osd.5 v2:172.21.15.196:6808/2766250253 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f586c01de50 con 0x7f58600ad970 2024-04-25T01:44:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.857+0000 7f587b8f5700 1 -- 172.21.15.78:0/2066358566 --> [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5874062dc0 con 0x7f58600ad970 2024-04-25T01:44:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.857+0000 7f586a7fc700 1 -- 172.21.15.78:0/2066358566 <== osd.5 v2:172.21.15.196:6808/2766250253 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5874062dc0 con 0x7f58600ad970 2024-04-25T01:44:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.858+0000 7f587b8f5700 1 -- 172.21.15.78:0/2066358566 >> [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253] conn(0x7f58600ad970 msgr2=0x7f58600afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.858+0000 7f587b8f5700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253] conn(0x7f58600ad970 0x7f58600afdb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.858+0000 7f587b8f5700 1 -- 172.21.15.78:0/2066358566 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f58600a2cb0 msgr2=0x7f58600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:02.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.858+0000 7f587b8f5700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f58600a2cb0 0x7f58600a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f58640099d0 tx=0x7f5864008040 comp rx=0 tx=0).stop 2024-04-25T01:44:02.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.858+0000 7f587b8f5700 1 -- 172.21.15.78:0/2066358566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58741082c0 msgr2=0x7f5874198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:02.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.858+0000 7f587b8f5700 1 --2- 172.21.15.78:0/2066358566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58741082c0 0x7f5874198840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f586c007ee0 tx=0x7f586c00d3b0 comp rx=0 tx=0).stop 2024-04-25T01:44:02.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.858+0000 7f5879e92700 1 -- 172.21.15.78:0/2066358566 reap_dead start 2024-04-25T01:44:02.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.858+0000 7f587b8f5700 1 -- 172.21.15.78:0/2066358566 shutdown_connections 2024-04-25T01:44:02.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.858+0000 7f587b8f5700 1 -- 172.21.15.78:0/2066358566 >> 172.21.15.78:0/2066358566 conn(0x7f58740777a0 msgr2=0x7f58740fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:02.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.858+0000 7f587b8f5700 1 -- 172.21.15.78:0/2066358566 shutdown_connections 2024-04-25T01:44:02.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:01.858+0000 7f587b8f5700 1 -- 172.21.15.78:0/2066358566 wait complete. 2024-04-25T01:44:04.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:04 smithi078 ceph-mon[29469]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T01:44:04.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:04 smithi152 ceph-mon[32465]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T01:44:04.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:04 smithi196 ceph-mon[32978]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T01:44:04.749 INFO:teuthology.orchestra.run.smithi078.stdout:167503724597 2024-04-25T01:44:04.750 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T01:44:06.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:06 smithi152 ceph-mon[32465]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:44:06.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:06 smithi196 ceph-mon[32978]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:44:06.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:06 smithi078 ceph-mon[29469]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:44:08.275 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:08 smithi078 ceph-mon[29469]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T01:44:08.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:08 smithi152 ceph-mon[32465]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T01:44:08.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:08 smithi196 ceph-mon[32978]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T01:44:09.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.350+0000 7f60dcd9a700 1 -- 172.21.15.78:0/3406955372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d8100820 msgr2=0x7f60d8100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:09.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.350+0000 7f60dcd9a700 1 --2- 172.21.15.78:0/3406955372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d8100820 0x7f60d8100c80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009990 tx=0x7f60cc009ca0 comp rx=0 tx=0).stop 2024-04-25T01:44:09.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.350+0000 7f60dcd9a700 1 -- 172.21.15.78:0/3406955372 shutdown_connections 2024-04-25T01:44:09.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.350+0000 7f60dcd9a700 1 --2- 172.21.15.78:0/3406955372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d81082c0 0x7f60d81086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:09.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.350+0000 7f60dcd9a700 1 --2- 172.21.15.78:0/3406955372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60d81011c0 0x7f60d810f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:09.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.350+0000 7f60dcd9a700 1 --2- 172.21.15.78:0/3406955372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d8100820 0x7f60d8100c80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:09.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.350+0000 7f60dcd9a700 1 -- 172.21.15.78:0/3406955372 >> 172.21.15.78:0/3406955372 conn(0x7f60d80777a0 msgr2=0x7f60d80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:09.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.350+0000 7f60dcd9a700 1 -- 172.21.15.78:0/3406955372 shutdown_connections 2024-04-25T01:44:09.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.350+0000 7f60dcd9a700 1 -- 172.21.15.78:0/3406955372 wait complete. 2024-04-25T01:44:09.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60dcd9a700 1 Processor -- start 2024-04-25T01:44:09.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60dcd9a700 1 -- start start 2024-04-25T01:44:09.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60dcd9a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d8100820 0x7f60d819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:09.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60dcd9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60d81011c0 0x7f60d819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:09.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60dcd9a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d81082c0 0x7f60d81987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:09.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60dcd9a700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60d8112520 con 0x7f60d8100820 2024-04-25T01:44:09.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60dcd9a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f60d81123a0 con 0x7f60d81011c0 2024-04-25T01:44:09.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60dcd9a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60d81126a0 con 0x7f60d81082c0 2024-04-25T01:44:09.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60d659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d8100820 0x7f60d819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:09.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60d659c700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d8100820 0x7f60d819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42684/0 (socket says 172.21.15.78:42684) 2024-04-25T01:44:09.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60d659c700 1 -- 172.21.15.78:0/2786023572 learned_addr learned my addr 172.21.15.78:0/2786023572 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:09.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60d6d9d700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d81082c0 0x7f60d81987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:09.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60d5d9b700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60d81011c0 0x7f60d819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:09.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60d659c700 1 -- 172.21.15.78:0/2786023572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d81082c0 msgr2=0x7f60d81987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:09.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60d659c700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60d81082c0 0x7f60d81987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:09.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60d659c700 1 -- 172.21.15.78:0/2786023572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60d81011c0 msgr2=0x7f60d819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:09.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60d659c700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60d81011c0 0x7f60d819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:09.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60d659c700 1 -- 172.21.15.78:0/2786023572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d8198f80 con 0x7f60d8100820 2024-04-25T01:44:09.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.351+0000 7f60d5d9b700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60d81011c0 0x7f60d819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:44:09.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.352+0000 7f60d659c700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d8100820 0x7f60d819e620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f60c800c920 tx=0x7f60c800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:09.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.352+0000 7f60c77fe700 1 -- 172.21.15.78:0/2786023572 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8007b00 con 0x7f60d8100820 2024-04-25T01:44:09.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.352+0000 7f60dcd9a700 1 -- 172.21.15.78:0/2786023572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d8199270 con 0x7f60d8100820 2024-04-25T01:44:09.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.352+0000 7f60c77fe700 1 -- 172.21.15.78:0/2786023572 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c8004d10 con 0x7f60d8100820 2024-04-25T01:44:09.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.352+0000 7f60c77fe700 1 -- 172.21.15.78:0/2786023572 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8005630 con 0x7f60d8100820 2024-04-25T01:44:09.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.352+0000 7f60dcd9a700 1 -- 172.21.15.78:0/2786023572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d81a5480 con 0x7f60d8100820 2024-04-25T01:44:09.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.353+0000 7f60dcd9a700 1 -- 172.21.15.78:0/2786023572 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f60d80629c0 con 0x7f60d8100820 2024-04-25T01:44:09.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.354+0000 7f60c77fe700 1 -- 172.21.15.78:0/2786023572 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f60c8004750 con 0x7f60d8100820 2024-04-25T01:44:09.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.355+0000 7f60c77fe700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f60c00a2cb0 0x7f60c00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:09.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.355+0000 7f60d5d9b700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f60c00a2cb0 0x7f60c00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:09.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.355+0000 7f60c77fe700 1 -- 172.21.15.78:0/2786023572 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f60c80cf340 con 0x7f60d8100820 2024-04-25T01:44:09.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.355+0000 7f60c77fe700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312] conn(0x7f60c00ad970 0x7f60c00afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:09.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.355+0000 7f60c77fe700 1 -- 172.21.15.78:0/2786023572 --> [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60c8022e50 con 0x7f60c00ad970 2024-04-25T01:44:09.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.355+0000 7f60c77fe700 1 -- 172.21.15.78:0/2786023572 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f60c80cf790 con 0x7f60d8100820 2024-04-25T01:44:09.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.355+0000 7f60d5d9b700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f60c00a2cb0 0x7f60c00a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00b5c0 tx=0x7f60cc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:09.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.355+0000 7f60d6d9d700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312] conn(0x7f60c00ad970 0x7f60c00afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:09.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.356+0000 7f60d6d9d700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312] conn(0x7f60c00ad970 0x7f60c00afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:09.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.357+0000 7f60c77fe700 1 -- 172.21.15.78:0/2786023572 <== osd.4 v2:172.21.15.152:6808/2556929312 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60c8022e50 con 0x7f60c00ad970 2024-04-25T01:44:09.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.373+0000 7f60dcd9a700 1 -- 172.21.15.78:0/2786023572 --> [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f60d8062dc0 con 0x7f60c00ad970 2024-04-25T01:44:09.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.374+0000 7f60c77fe700 1 -- 172.21.15.78:0/2786023572 <== osd.4 v2:172.21.15.152:6808/2556929312 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f60d8062dc0 con 0x7f60c00ad970 2024-04-25T01:44:09.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.374+0000 7f60dcd9a700 1 -- 172.21.15.78:0/2786023572 >> [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312] conn(0x7f60c00ad970 msgr2=0x7f60c00afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:09.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.374+0000 7f60dcd9a700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312] conn(0x7f60c00ad970 0x7f60c00afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:09.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.374+0000 7f60dcd9a700 1 -- 172.21.15.78:0/2786023572 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f60c00a2cb0 msgr2=0x7f60c00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:09.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.374+0000 7f60dcd9a700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f60c00a2cb0 0x7f60c00a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00b5c0 tx=0x7f60cc005fb0 comp rx=0 tx=0).stop 2024-04-25T01:44:09.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.374+0000 7f60dcd9a700 1 -- 172.21.15.78:0/2786023572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d8100820 msgr2=0x7f60d819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:09.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.374+0000 7f60dcd9a700 1 --2- 172.21.15.78:0/2786023572 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60d8100820 0x7f60d819e620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f60c800c920 tx=0x7f60c800cc30 comp rx=0 tx=0).stop 2024-04-25T01:44:09.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.374+0000 7f60d6d9d700 1 -- 172.21.15.78:0/2786023572 reap_dead start 2024-04-25T01:44:09.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.374+0000 7f60dcd9a700 1 -- 172.21.15.78:0/2786023572 shutdown_connections 2024-04-25T01:44:09.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.374+0000 7f60dcd9a700 1 -- 172.21.15.78:0/2786023572 >> 172.21.15.78:0/2786023572 conn(0x7f60d80777a0 msgr2=0x7f60d80feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:09.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.374+0000 7f60dcd9a700 1 -- 172.21.15.78:0/2786023572 shutdown_connections 2024-04-25T01:44:09.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:09.374+0000 7f60dcd9a700 1 -- 172.21.15.78:0/2786023572 wait complete. 2024-04-25T01:44:10.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:10 smithi078 ceph-mon[29469]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:44:10.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:10 smithi152 ceph-mon[32465]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:44:10.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:10 smithi196 ceph-mon[32978]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:44:12.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:12 smithi078 ceph-mon[29469]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:44:12.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:12 smithi152 ceph-mon[32465]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:44:12.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:12 smithi196 ceph-mon[32978]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:44:13.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.098+0000 7fd439ac0700 1 -- 172.21.15.78:0/526938614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd434100820 msgr2=0x7fd434100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.098+0000 7fd439ac0700 1 --2- 172.21.15.78:0/526938614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd434100820 0x7fd434100c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd424009990 tx=0x7fd424009ca0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.098+0000 7fd439ac0700 1 -- 172.21.15.78:0/526938614 shutdown_connections 2024-04-25T01:44:13.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.098+0000 7fd439ac0700 1 --2- 172.21.15.78:0/526938614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4341011c0 0x7fd43410f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.098+0000 7fd439ac0700 1 --2- 172.21.15.78:0/526938614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4341082c0 0x7fd4341086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.098+0000 7fd439ac0700 1 --2- 172.21.15.78:0/526938614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd434100820 0x7fd434100c80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.098+0000 7fd439ac0700 1 -- 172.21.15.78:0/526938614 >> 172.21.15.78:0/526938614 conn(0x7fd4340777a0 msgr2=0x7fd4340feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:13.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.098+0000 7fd439ac0700 1 -- 172.21.15.78:0/526938614 shutdown_connections 2024-04-25T01:44:13.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.098+0000 7fd439ac0700 1 -- 172.21.15.78:0/526938614 wait complete. 2024-04-25T01:44:13.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.099+0000 7fd439ac0700 1 Processor -- start 2024-04-25T01:44:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.099+0000 7fd439ac0700 1 -- start start 2024-04-25T01:44:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.099+0000 7fd439ac0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd434100820 0x7fd43419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.099+0000 7fd439ac0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4341011c0 0x7fd43419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.099+0000 7fd439ac0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4341082c0 0x7fd434198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.099+0000 7fd439ac0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd434112580 con 0x7fd4341011c0 2024-04-25T01:44:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.099+0000 7fd439ac0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd434112400 con 0x7fd434100820 2024-04-25T01:44:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.099+0000 7fd439ac0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd434112700 con 0x7fd4341082c0 2024-04-25T01:44:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.099+0000 7fd432ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4341011c0 0x7fd43419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.099+0000 7fd432ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4341011c0 0x7fd43419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44788/0 (socket says 172.21.15.78:44788) 2024-04-25T01:44:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.099+0000 7fd432ffd700 1 -- 172.21.15.78:0/3071862917 learned_addr learned my addr 172.21.15.78:0/3071862917 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.099+0000 7fd4337fe700 1 --2- 172.21.15.78:0/3071862917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd434100820 0x7fd43419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.100+0000 7fd433fff700 1 --2- 172.21.15.78:0/3071862917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4341082c0 0x7fd434198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.100+0000 7fd432ffd700 1 -- 172.21.15.78:0/3071862917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4341082c0 msgr2=0x7fd434198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.100+0000 7fd432ffd700 1 --2- 172.21.15.78:0/3071862917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4341082c0 0x7fd434198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.100+0000 7fd432ffd700 1 -- 172.21.15.78:0/3071862917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd434100820 msgr2=0x7fd43419e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.100+0000 7fd432ffd700 1 --2- 172.21.15.78:0/3071862917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd434100820 0x7fd43419e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.100+0000 7fd432ffd700 1 -- 172.21.15.78:0/3071862917 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd434198fe0 con 0x7fd4341011c0 2024-04-25T01:44:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.100+0000 7fd432ffd700 1 --2- 172.21.15.78:0/3071862917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4341011c0 0x7fd43419ebc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd42400bff0 tx=0x7fd424005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.100+0000 7fd430ff9700 1 -- 172.21.15.78:0/3071862917 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424027070 con 0x7fd4341011c0 2024-04-25T01:44:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.100+0000 7fd430ff9700 1 -- 172.21.15.78:0/3071862917 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd424004050 con 0x7fd4341011c0 2024-04-25T01:44:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.100+0000 7fd439ac0700 1 -- 172.21.15.78:0/3071862917 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd434199270 con 0x7fd4341011c0 2024-04-25T01:44:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.100+0000 7fd430ff9700 1 -- 172.21.15.78:0/3071862917 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424022d00 con 0x7fd4341011c0 2024-04-25T01:44:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.100+0000 7fd439ac0700 1 -- 172.21.15.78:0/3071862917 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4341a54e0 con 0x7fd4341011c0 2024-04-25T01:44:13.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.101+0000 7fd439ac0700 1 -- 172.21.15.78:0/3071862917 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd4340629c0 con 0x7fd4341011c0 2024-04-25T01:44:13.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.102+0000 7fd430ff9700 1 -- 172.21.15.78:0/3071862917 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd42402a070 con 0x7fd4341011c0 2024-04-25T01:44:13.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.103+0000 7fd430ff9700 1 --2- 172.21.15.78:0/3071862917 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd41c0a2d30 0x7fd41c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.103+0000 7fd4337fe700 1 --2- 172.21.15.78:0/3071862917 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd41c0a2d30 0x7fd41c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.103+0000 7fd430ff9700 1 -- 172.21.15.78:0/3071862917 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd4240d0d80 con 0x7fd4341011c0 2024-04-25T01:44:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.103+0000 7fd430ff9700 1 --2- 172.21.15.78:0/3071862917 >> [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426] conn(0x7fd41c0ad9f0 0x7fd41c0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.103+0000 7fd430ff9700 1 -- 172.21.15.78:0/3071862917 --> [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd424001d30 con 0x7fd41c0ad9f0 2024-04-25T01:44:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.103+0000 7fd430ff9700 1 -- 172.21.15.78:0/3071862917 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fd4240d11d0 con 0x7fd4341011c0 2024-04-25T01:44:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.103+0000 7fd4337fe700 1 --2- 172.21.15.78:0/3071862917 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd41c0a2d30 0x7fd41c0a51f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd428009a10 tx=0x7fd428009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.103+0000 7fd433fff700 1 --2- 172.21.15.78:0/3071862917 >> [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426] conn(0x7fd41c0ad9f0 0x7fd41c0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.104+0000 7fd433fff700 1 --2- 172.21.15.78:0/3071862917 >> [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426] conn(0x7fd41c0ad9f0 0x7fd41c0afe30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.105+0000 7fd430ff9700 1 -- 172.21.15.78:0/3071862917 <== osd.8 v2:172.21.15.196:6816/3733356426 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd424001d30 con 0x7fd41c0ad9f0 2024-04-25T01:44:13.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.121+0000 7fd439ac0700 1 -- 172.21.15.78:0/3071862917 --> [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd434062dc0 con 0x7fd41c0ad9f0 2024-04-25T01:44:13.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.121+0000 7fd430ff9700 1 -- 172.21.15.78:0/3071862917 <== osd.8 v2:172.21.15.196:6816/3733356426 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd434062dc0 con 0x7fd41c0ad9f0 2024-04-25T01:44:13.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.122+0000 7fd439ac0700 1 -- 172.21.15.78:0/3071862917 >> [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426] conn(0x7fd41c0ad9f0 msgr2=0x7fd41c0afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.122+0000 7fd439ac0700 1 --2- 172.21.15.78:0/3071862917 >> [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426] conn(0x7fd41c0ad9f0 0x7fd41c0afe30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.122+0000 7fd439ac0700 1 -- 172.21.15.78:0/3071862917 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd41c0a2d30 msgr2=0x7fd41c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.122+0000 7fd439ac0700 1 --2- 172.21.15.78:0/3071862917 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd41c0a2d30 0x7fd41c0a51f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd428009a10 tx=0x7fd428009190 comp rx=0 tx=0).stop 2024-04-25T01:44:13.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.122+0000 7fd439ac0700 1 -- 172.21.15.78:0/3071862917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4341011c0 msgr2=0x7fd43419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.122+0000 7fd439ac0700 1 --2- 172.21.15.78:0/3071862917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4341011c0 0x7fd43419ebc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd42400bff0 tx=0x7fd424005590 comp rx=0 tx=0).stop 2024-04-25T01:44:13.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.122+0000 7fd433fff700 1 -- 172.21.15.78:0/3071862917 reap_dead start 2024-04-25T01:44:13.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.122+0000 7fd439ac0700 1 -- 172.21.15.78:0/3071862917 shutdown_connections 2024-04-25T01:44:13.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.122+0000 7fd439ac0700 1 -- 172.21.15.78:0/3071862917 >> 172.21.15.78:0/3071862917 conn(0x7fd4340777a0 msgr2=0x7fd4340feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:13.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.122+0000 7fd439ac0700 1 -- 172.21.15.78:0/3071862917 shutdown_connections 2024-04-25T01:44:13.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.122+0000 7fd439ac0700 1 -- 172.21.15.78:0/3071862917 wait complete. 2024-04-25T01:44:13.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.591+0000 7ff15aa6e700 1 -- 172.21.15.78:0/2036279687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff154107570 msgr2=0x7ff15410f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.591+0000 7ff15aa6e700 1 --2- 172.21.15.78:0/2036279687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff154107570 0x7ff15410f140 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00b0a0 tx=0x7ff14c00b3b0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.591+0000 7ff15aa6e700 1 -- 172.21.15.78:0/2036279687 shutdown_connections 2024-04-25T01:44:13.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.591+0000 7ff15aa6e700 1 --2- 172.21.15.78:0/2036279687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff154107570 0x7ff15410f140 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.591+0000 7ff15aa6e700 1 --2- 172.21.15.78:0/2036279687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1541024e0 0x7ff1541028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.591+0000 7ff15aa6e700 1 --2- 172.21.15.78:0/2036279687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff154102e90 0x7ff154106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.591+0000 7ff15aa6e700 1 -- 172.21.15.78:0/2036279687 >> 172.21.15.78:0/2036279687 conn(0x7ff154079390 msgr2=0x7ff1540797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:13.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.591+0000 7ff15aa6e700 1 -- 172.21.15.78:0/2036279687 shutdown_connections 2024-04-25T01:44:13.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.592+0000 7ff15aa6e700 1 -- 172.21.15.78:0/2036279687 wait complete. 2024-04-25T01:44:13.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.592+0000 7ff15aa6e700 1 Processor -- start 2024-04-25T01:44:13.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.592+0000 7ff15aa6e700 1 -- start start 2024-04-25T01:44:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.592+0000 7ff15aa6e700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1541024e0 0x7ff15419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.592+0000 7ff15aa6e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff154102e90 0x7ff15419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.592+0000 7ff15aa6e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff154107570 0x7ff1541987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.592+0000 7ff15aa6e700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff154112550 con 0x7ff1541024e0 2024-04-25T01:44:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.592+0000 7ff15aa6e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff1541123d0 con 0x7ff154102e90 2024-04-25T01:44:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.592+0000 7ff15aa6e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1541126d0 con 0x7ff154107570 2024-04-25T01:44:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.592+0000 7ff15880a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1541024e0 0x7ff15419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.592+0000 7ff15880a700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1541024e0 0x7ff15419e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44794/0 (socket says 172.21.15.78:44794) 2024-04-25T01:44:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.592+0000 7ff15880a700 1 -- 172.21.15.78:0/1472312989 learned_addr learned my addr 172.21.15.78:0/1472312989 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.593+0000 7ff153fff700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff154102e90 0x7ff15419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.593+0000 7ff15900b700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff154107570 0x7ff1541987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.593+0000 7ff15880a700 1 -- 172.21.15.78:0/1472312989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff154107570 msgr2=0x7ff1541987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.593+0000 7ff15880a700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff154107570 0x7ff1541987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.593+0000 7ff15880a700 1 -- 172.21.15.78:0/1472312989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff154102e90 msgr2=0x7ff15419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.593+0000 7ff15880a700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff154102e90 0x7ff15419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.593+0000 7ff15880a700 1 -- 172.21.15.78:0/1472312989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff154198fd0 con 0x7ff1541024e0 2024-04-25T01:44:13.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.593+0000 7ff153fff700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff154102e90 0x7ff15419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:44:13.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.593+0000 7ff15880a700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1541024e0 0x7ff15419e670 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff14800d950 tx=0x7ff14800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.593+0000 7ff151ffb700 1 -- 172.21.15.78:0/1472312989 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148016070 con 0x7ff1541024e0 2024-04-25T01:44:13.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.593+0000 7ff15aa6e700 1 -- 172.21.15.78:0/1472312989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1541992c0 con 0x7ff1541024e0 2024-04-25T01:44:13.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.593+0000 7ff151ffb700 1 -- 172.21.15.78:0/1472312989 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff148009b40 con 0x7ff1541024e0 2024-04-25T01:44:13.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.593+0000 7ff151ffb700 1 -- 172.21.15.78:0/1472312989 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14800f620 con 0x7ff1541024e0 2024-04-25T01:44:13.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.593+0000 7ff15aa6e700 1 -- 172.21.15.78:0/1472312989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1541a54d0 con 0x7ff1541024e0 2024-04-25T01:44:13.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.594+0000 7ff15aa6e700 1 -- 172.21.15.78:0/1472312989 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff1540629c0 con 0x7ff1541024e0 2024-04-25T01:44:13.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.595+0000 7ff151ffb700 1 -- 172.21.15.78:0/1472312989 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff148019070 con 0x7ff1541024e0 2024-04-25T01:44:13.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.596+0000 7ff151ffb700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff1440a2cb0 0x7ff1440a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.596+0000 7ff151ffb700 1 -- 172.21.15.78:0/1472312989 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff1480cf450 con 0x7ff1541024e0 2024-04-25T01:44:13.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.596+0000 7ff153fff700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff1440a2cb0 0x7ff1440a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.596+0000 7ff151ffb700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986] conn(0x7ff1440ad970 0x7ff1440afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.596+0000 7ff151ffb700 1 -- 172.21.15.78:0/1472312989 --> [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1440b0480 con 0x7ff1440ad970 2024-04-25T01:44:13.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.596+0000 7ff151ffb700 1 -- 172.21.15.78:0/1472312989 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff1480d6050 con 0x7ff1541024e0 2024-04-25T01:44:13.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.596+0000 7ff15900b700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986] conn(0x7ff1440ad970 0x7ff1440afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.596+0000 7ff153fff700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff1440a2cb0 0x7ff1440a5170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff140005e10 tx=0x7ff140005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.597+0000 7ff15900b700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986] conn(0x7ff1440ad970 0x7ff1440afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.597+0000 7ff151ffb700 1 -- 172.21.15.78:0/1472312989 <== osd.11 v2:172.21.15.196:6824/3010681986 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1440b0480 con 0x7ff1440ad970 2024-04-25T01:44:13.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.614+0000 7ff15aa6e700 1 -- 172.21.15.78:0/1472312989 --> [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff154062dc0 con 0x7ff1440ad970 2024-04-25T01:44:13.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.614+0000 7ff151ffb700 1 -- 172.21.15.78:0/1472312989 <== osd.11 v2:172.21.15.196:6824/3010681986 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff154062dc0 con 0x7ff1440ad970 2024-04-25T01:44:13.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.614+0000 7ff15aa6e700 1 -- 172.21.15.78:0/1472312989 >> [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986] conn(0x7ff1440ad970 msgr2=0x7ff1440afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.614+0000 7ff15aa6e700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986] conn(0x7ff1440ad970 0x7ff1440afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.615+0000 7ff15aa6e700 1 -- 172.21.15.78:0/1472312989 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff1440a2cb0 msgr2=0x7ff1440a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.615+0000 7ff15aa6e700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff1440a2cb0 0x7ff1440a5170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff140005e10 tx=0x7ff140005d80 comp rx=0 tx=0).stop 2024-04-25T01:44:13.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.615+0000 7ff15aa6e700 1 -- 172.21.15.78:0/1472312989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1541024e0 msgr2=0x7ff15419e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.615+0000 7ff15aa6e700 1 --2- 172.21.15.78:0/1472312989 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1541024e0 0x7ff15419e670 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff14800d950 tx=0x7ff14800dd10 comp rx=0 tx=0).stop 2024-04-25T01:44:13.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.615+0000 7ff15900b700 1 -- 172.21.15.78:0/1472312989 reap_dead start 2024-04-25T01:44:13.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.615+0000 7ff15aa6e700 1 -- 172.21.15.78:0/1472312989 shutdown_connections 2024-04-25T01:44:13.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.615+0000 7ff15aa6e700 1 -- 172.21.15.78:0/1472312989 >> 172.21.15.78:0/1472312989 conn(0x7ff154079390 msgr2=0x7ff154100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:13.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.615+0000 7ff15aa6e700 1 -- 172.21.15.78:0/1472312989 shutdown_connections 2024-04-25T01:44:13.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.615+0000 7ff15aa6e700 1 -- 172.21.15.78:0/1472312989 wait complete. 2024-04-25T01:44:13.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.801+0000 7f6b920f4700 1 -- 172.21.15.78:0/1962283120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c101ce0 msgr2=0x7f6b8c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.801+0000 7f6b920f4700 1 --2- 172.21.15.78:0/1962283120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c101ce0 0x7f6b8c102160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6b740099c0 tx=0x7f6b74009cd0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.801+0000 7f6b920f4700 1 -- 172.21.15.78:0/1962283120 shutdown_connections 2024-04-25T01:44:13.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.801+0000 7f6b920f4700 1 --2- 172.21.15.78:0/1962283120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b8c107400 0x7f6b8c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.801+0000 7f6b920f4700 1 --2- 172.21.15.78:0/1962283120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b8c1026a0 0x7f6b8c10e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.801+0000 7f6b920f4700 1 --2- 172.21.15.78:0/1962283120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c101ce0 0x7f6b8c102160 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.801+0000 7f6b920f4700 1 -- 172.21.15.78:0/1962283120 >> 172.21.15.78:0/1962283120 conn(0x7f6b8c076450 msgr2=0x7f6b8c076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:13.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.801+0000 7f6b920f4700 1 -- 172.21.15.78:0/1962283120 shutdown_connections 2024-04-25T01:44:13.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.801+0000 7f6b920f4700 1 -- 172.21.15.78:0/1962283120 wait complete. 2024-04-25T01:44:13.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b920f4700 1 Processor -- start 2024-04-25T01:44:13.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b920f4700 1 -- start start 2024-04-25T01:44:13.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b920f4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c101ce0 0x7f6b8c197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b920f4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b8c1026a0 0x7f6b8c197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b920f4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b8c107400 0x7f6b8c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b920f4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c111570 con 0x7f6b8c101ce0 2024-04-25T01:44:13.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b920f4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1113f0 con 0x7f6b8c107400 2024-04-25T01:44:13.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b920f4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1116f0 con 0x7f6b8c1026a0 2024-04-25T01:44:13.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b8b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c101ce0 0x7f6b8c197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b8b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c101ce0 0x7f6b8c197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44804/0 (socket says 172.21.15.78:44804) 2024-04-25T01:44:13.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b8b7fe700 1 -- 172.21.15.78:0/4198871946 learned_addr learned my addr 172.21.15.78:0/4198871946 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:13.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b83fff700 1 --2- 172.21.15.78:0/4198871946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b8c1026a0 0x7f6b8c197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b8bfff700 1 --2- 172.21.15.78:0/4198871946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b8c107400 0x7f6b8c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b8b7fe700 1 -- 172.21.15.78:0/4198871946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b8c1026a0 msgr2=0x7f6b8c197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b8b7fe700 1 --2- 172.21.15.78:0/4198871946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b8c1026a0 0x7f6b8c197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b8b7fe700 1 -- 172.21.15.78:0/4198871946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b8c107400 msgr2=0x7f6b8c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b8b7fe700 1 --2- 172.21.15.78:0/4198871946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b8c107400 0x7f6b8c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.802+0000 7f6b8b7fe700 1 -- 172.21.15.78:0/4198871946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c1a7000 con 0x7f6b8c101ce0 2024-04-25T01:44:13.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.803+0000 7f6b8b7fe700 1 --2- 172.21.15.78:0/4198871946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c101ce0 0x7f6b8c197700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00c970 tx=0x7f6b7c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.803+0000 7f6b897fa700 1 -- 172.21.15.78:0/4198871946 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c007aa0 con 0x7f6b8c101ce0 2024-04-25T01:44:13.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.803+0000 7f6b897fa700 1 -- 172.21.15.78:0/4198871946 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b7c004d10 con 0x7f6b8c101ce0 2024-04-25T01:44:13.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.803+0000 7f6b920f4700 1 -- 172.21.15.78:0/4198871946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8c1a72f0 con 0x7f6b8c101ce0 2024-04-25T01:44:13.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.803+0000 7f6b897fa700 1 -- 172.21.15.78:0/4198871946 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00f450 con 0x7f6b8c101ce0 2024-04-25T01:44:13.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.803+0000 7f6b920f4700 1 -- 172.21.15.78:0/4198871946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8c1a77e0 con 0x7f6b8c101ce0 2024-04-25T01:44:13.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.803+0000 7f6b920f4700 1 -- 172.21.15.78:0/4198871946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b8c062940 con 0x7f6b8c101ce0 2024-04-25T01:44:13.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.805+0000 7f6b897fa700 1 -- 172.21.15.78:0/4198871946 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6b7c004750 con 0x7f6b8c101ce0 2024-04-25T01:44:13.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.805+0000 7f6b897fa700 1 --2- 172.21.15.78:0/4198871946 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6b780a2cb0 0x7f6b780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.805+0000 7f6b83fff700 1 --2- 172.21.15.78:0/4198871946 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6b780a2cb0 0x7f6b780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.805+0000 7f6b897fa700 1 -- 172.21.15.78:0/4198871946 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6b7c0cf610 con 0x7f6b8c101ce0 2024-04-25T01:44:13.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.806+0000 7f6b897fa700 1 --2- 172.21.15.78:0/4198871946 >> [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277] conn(0x7f6b780ad970 0x7f6b780afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.806+0000 7f6b897fa700 1 -- 172.21.15.78:0/4198871946 --> [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b7c016ec0 con 0x7f6b780ad970 2024-04-25T01:44:13.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.806+0000 7f6b897fa700 1 -- 172.21.15.78:0/4198871946 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f6b7c0cfa60 con 0x7f6b8c101ce0 2024-04-25T01:44:13.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.806+0000 7f6b8bfff700 1 --2- 172.21.15.78:0/4198871946 >> [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277] conn(0x7f6b780ad970 0x7f6b780afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.806+0000 7f6b83fff700 1 --2- 172.21.15.78:0/4198871946 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6b780a2cb0 0x7f6b780a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c199860 tx=0x7f6b74005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.806+0000 7f6b8bfff700 1 --2- 172.21.15.78:0/4198871946 >> [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277] conn(0x7f6b780ad970 0x7f6b780afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.807+0000 7f6b897fa700 1 -- 172.21.15.78:0/4198871946 <== osd.9 v2:172.21.15.78:6826/3327461277 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b7c016ec0 con 0x7f6b780ad970 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.823+0000 7f6b920f4700 1 -- 172.21.15.78:0/4198871946 --> [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6b8c062d40 con 0x7f6b780ad970 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.823+0000 7f6b897fa700 1 -- 172.21.15.78:0/4198871946 <== osd.9 v2:172.21.15.78:6826/3327461277 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6b8c062d40 con 0x7f6b780ad970 2024-04-25T01:44:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.823+0000 7f6b920f4700 1 -- 172.21.15.78:0/4198871946 >> [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277] conn(0x7f6b780ad970 msgr2=0x7f6b780afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.823+0000 7f6b920f4700 1 --2- 172.21.15.78:0/4198871946 >> [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277] conn(0x7f6b780ad970 0x7f6b780afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.823+0000 7f6b920f4700 1 -- 172.21.15.78:0/4198871946 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6b780a2cb0 msgr2=0x7f6b780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.823+0000 7f6b920f4700 1 --2- 172.21.15.78:0/4198871946 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6b780a2cb0 0x7f6b780a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c199860 tx=0x7f6b74005fb0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.823+0000 7f6b920f4700 1 -- 172.21.15.78:0/4198871946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c101ce0 msgr2=0x7f6b8c197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.823+0000 7f6b920f4700 1 --2- 172.21.15.78:0/4198871946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b8c101ce0 0x7f6b8c197700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00c970 tx=0x7f6b7c00cc80 comp rx=0 tx=0).stop 2024-04-25T01:44:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.824+0000 7f6b8bfff700 1 -- 172.21.15.78:0/4198871946 reap_dead start 2024-04-25T01:44:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.824+0000 7f6b920f4700 1 -- 172.21.15.78:0/4198871946 shutdown_connections 2024-04-25T01:44:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.824+0000 7f6b920f4700 1 -- 172.21.15.78:0/4198871946 >> 172.21.15.78:0/4198871946 conn(0x7f6b8c076450 msgr2=0x7f6b8c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.824+0000 7f6b920f4700 1 -- 172.21.15.78:0/4198871946 shutdown_connections 2024-04-25T01:44:13.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:13.824+0000 7f6b920f4700 1 -- 172.21.15.78:0/4198871946 wait complete. 2024-04-25T01:44:14.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:14 smithi152 ceph-mon[32465]: pgmap v318: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T01:44:14.362 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:14 smithi078 ceph-mon[29469]: pgmap v318: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T01:44:14.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:14 smithi196 ceph-mon[32978]: pgmap v318: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T01:44:14.995 INFO:teuthology.orchestra.run.smithi078.stdout:141733920828 2024-04-25T01:44:14.996 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T01:44:15.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:15 smithi078 ceph-mon[29469]: pgmap v319: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T01:44:15.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:15 smithi196 ceph-mon[32978]: pgmap v319: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T01:44:15.454 INFO:teuthology.orchestra.run.smithi078.stdout:347892351001 2024-04-25T01:44:15.455 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T01:44:15.548 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:15 smithi152 ceph-mon[32465]: pgmap v319: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T01:44:15.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.654+0000 7f1146d7d700 1 -- 172.21.15.78:0/3827019174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1140100820 msgr2=0x7f1140100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:15.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.654+0000 7f1146d7d700 1 --2- 172.21.15.78:0/3827019174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1140100820 0x7f1140100c80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f112c009990 tx=0x7f112c009ca0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.654+0000 7f1146d7d700 1 -- 172.21.15.78:0/3827019174 shutdown_connections 2024-04-25T01:44:15.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.654+0000 7f1146d7d700 1 --2- 172.21.15.78:0/3827019174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11401082c0 0x7f11401086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.654+0000 7f1146d7d700 1 --2- 172.21.15.78:0/3827019174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11401011c0 0x7f114010f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.654+0000 7f1146d7d700 1 --2- 172.21.15.78:0/3827019174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1140100820 0x7f1140100c80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.654+0000 7f1146d7d700 1 -- 172.21.15.78:0/3827019174 >> 172.21.15.78:0/3827019174 conn(0x7f11400777a0 msgr2=0x7f11400feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:15.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.654+0000 7f1146d7d700 1 -- 172.21.15.78:0/3827019174 shutdown_connections 2024-04-25T01:44:15.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.654+0000 7f1146d7d700 1 -- 172.21.15.78:0/3827019174 wait complete. 2024-04-25T01:44:15.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f1146d7d700 1 Processor -- start 2024-04-25T01:44:15.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f1146d7d700 1 -- start start 2024-04-25T01:44:15.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f1146d7d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1140100820 0x7f114019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:15.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f1146d7d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11401011c0 0x7f114019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:15.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f1146d7d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11401082c0 0x7f1140198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:15.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f1146d7d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1140112520 con 0x7f11401011c0 2024-04-25T01:44:15.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f1146d7d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f11401123a0 con 0x7f1140100820 2024-04-25T01:44:15.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f1146d7d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f11401126a0 con 0x7f11401082c0 2024-04-25T01:44:15.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f113ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11401011c0 0x7f114019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:15.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f113ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11401011c0 0x7f114019ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44822/0 (socket says 172.21.15.78:44822) 2024-04-25T01:44:15.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f113ffff700 1 -- 172.21.15.78:0/1037681272 learned_addr learned my addr 172.21.15.78:0/1037681272 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:15.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f1144b19700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1140100820 0x7f114019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:15.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f114531a700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11401082c0 0x7f1140198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:15.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f113ffff700 1 -- 172.21.15.78:0/1037681272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11401082c0 msgr2=0x7f1140198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f113ffff700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11401082c0 0x7f1140198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f113ffff700 1 -- 172.21.15.78:0/1037681272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1140100820 msgr2=0x7f114019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f113ffff700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1140100820 0x7f114019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f113ffff700 1 -- 172.21.15.78:0/1037681272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1140199040 con 0x7f11401011c0 2024-04-25T01:44:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.655+0000 7f114531a700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11401082c0 0x7f1140198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:44:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.656+0000 7f113ffff700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11401011c0 0x7f114019ebd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f112c000c00 tx=0x7f112c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.656+0000 7f113dffb700 1 -- 172.21.15.78:0/1037681272 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c027070 con 0x7f11401011c0 2024-04-25T01:44:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.656+0000 7f113dffb700 1 -- 172.21.15.78:0/1037681272 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f112c004190 con 0x7f11401011c0 2024-04-25T01:44:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.656+0000 7f1146d7d700 1 -- 172.21.15.78:0/1037681272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11401992d0 con 0x7f11401011c0 2024-04-25T01:44:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.656+0000 7f113dffb700 1 -- 172.21.15.78:0/1037681272 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c00f460 con 0x7f11401011c0 2024-04-25T01:44:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.656+0000 7f1146d7d700 1 -- 172.21.15.78:0/1037681272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11401a5440 con 0x7f11401011c0 2024-04-25T01:44:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.657+0000 7f1146d7d700 1 -- 172.21.15.78:0/1037681272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1140062980 con 0x7f11401011c0 2024-04-25T01:44:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.658+0000 7f113dffb700 1 -- 172.21.15.78:0/1037681272 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f112c02a070 con 0x7f11401011c0 2024-04-25T01:44:15.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.659+0000 7f113dffb700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f11300a2cb0 0x7f11300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.659+0000 7f113dffb700 1 -- 172.21.15.78:0/1037681272 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f112c0d0b10 con 0x7f11401011c0 2024-04-25T01:44:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.659+0000 7f1144b19700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f11300a2cb0 0x7f11300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.659+0000 7f113dffb700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494] conn(0x7f11300ad970 0x7f11300afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.659+0000 7f113dffb700 1 -- 172.21.15.78:0/1037681272 --> [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f112c001d30 con 0x7f11300ad970 2024-04-25T01:44:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.659+0000 7f113dffb700 1 -- 172.21.15.78:0/1037681272 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f112c0d0f60 con 0x7f11401011c0 2024-04-25T01:44:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.659+0000 7f1144b19700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f11300a2cb0 0x7f11300a5170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1134005fd0 tx=0x7f1134005db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.659+0000 7f114531a700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494] conn(0x7f11300ad970 0x7f11300afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.660+0000 7f114531a700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494] conn(0x7f11300ad970 0x7f11300afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:15.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.661+0000 7f113dffb700 1 -- 172.21.15.78:0/1037681272 <== osd.1 v2:172.21.15.152:6800/2731006494 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f112c001d30 con 0x7f11300ad970 2024-04-25T01:44:15.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.677+0000 7f1146d7d700 1 -- 172.21.15.78:0/1037681272 --> [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1140062e00 con 0x7f11300ad970 2024-04-25T01:44:15.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.678+0000 7f113dffb700 1 -- 172.21.15.78:0/1037681272 <== osd.1 v2:172.21.15.152:6800/2731006494 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1140062e00 con 0x7f11300ad970 2024-04-25T01:44:15.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.678+0000 7f1146d7d700 1 -- 172.21.15.78:0/1037681272 >> [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494] conn(0x7f11300ad970 msgr2=0x7f11300afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:15.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.678+0000 7f1146d7d700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494] conn(0x7f11300ad970 0x7f11300afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.678+0000 7f1146d7d700 1 -- 172.21.15.78:0/1037681272 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f11300a2cb0 msgr2=0x7f11300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:15.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.678+0000 7f1146d7d700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f11300a2cb0 0x7f11300a5170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1134005fd0 tx=0x7f1134005db0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.678+0000 7f1146d7d700 1 -- 172.21.15.78:0/1037681272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11401011c0 msgr2=0x7f114019ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:15.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.678+0000 7f1146d7d700 1 --2- 172.21.15.78:0/1037681272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11401011c0 0x7f114019ebd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f112c000c00 tx=0x7f112c003730 comp rx=0 tx=0).stop 2024-04-25T01:44:15.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.678+0000 7f114531a700 1 -- 172.21.15.78:0/1037681272 reap_dead start 2024-04-25T01:44:15.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.678+0000 7f1146d7d700 1 -- 172.21.15.78:0/1037681272 shutdown_connections 2024-04-25T01:44:15.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.678+0000 7f1146d7d700 1 -- 172.21.15.78:0/1037681272 >> 172.21.15.78:0/1037681272 conn(0x7f11400777a0 msgr2=0x7f11400fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:15.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.678+0000 7f1146d7d700 1 -- 172.21.15.78:0/1037681272 shutdown_connections 2024-04-25T01:44:15.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:15.678+0000 7f1146d7d700 1 -- 172.21.15.78:0/1037681272 wait complete. 2024-04-25T01:44:16.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:16 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:44:16.600 INFO:teuthology.orchestra.run.smithi078.stdout:257698037800 2024-04-25T01:44:16.601 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T01:44:16.798 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:16 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:44:16.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:16 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:44:17.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.280+0000 7f61180c1700 1 -- 172.21.15.78:0/3965068358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61101082b0 msgr2=0x7f6110108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:17.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.280+0000 7f61180c1700 1 --2- 172.21.15.78:0/3965068358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61101082b0 0x7f6110108690 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6100009990 tx=0x7f6100009ca0 comp rx=0 tx=0).stop 2024-04-25T01:44:17.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.281+0000 7f61180c1700 1 -- 172.21.15.78:0/3965068358 shutdown_connections 2024-04-25T01:44:17.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.281+0000 7f61180c1700 1 --2- 172.21.15.78:0/3965068358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f611006a8e0 0x7f611010f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:17.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.281+0000 7f61180c1700 1 --2- 172.21.15.78:0/3965068358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61101082b0 0x7f6110108690 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:17.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.281+0000 7f61180c1700 1 --2- 172.21.15.78:0/3965068358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6110069f40 0x7f611006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:17.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.281+0000 7f61180c1700 1 -- 172.21.15.78:0/3965068358 >> 172.21.15.78:0/3965068358 conn(0x7f61100780e0 msgr2=0x7f61100784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:17.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.281+0000 7f61180c1700 1 -- 172.21.15.78:0/3965068358 shutdown_connections 2024-04-25T01:44:17.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.281+0000 7f61180c1700 1 -- 172.21.15.78:0/3965068358 wait complete. 2024-04-25T01:44:17.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.281+0000 7f61180c1700 1 Processor -- start 2024-04-25T01:44:17.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.281+0000 7f61180c1700 1 -- start start 2024-04-25T01:44:17.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f61180c1700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6110069f40 0x7f611019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:17.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f61180c1700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f611006a8e0 0x7f611019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:17.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f61180c1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61101082b0 0x7f61101988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:17.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f61180c1700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6110112560 con 0x7f6110069f40 2024-04-25T01:44:17.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f61180c1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f61101123e0 con 0x7f61101082b0 2024-04-25T01:44:17.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f61180c1700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61101126e0 con 0x7f611006a8e0 2024-04-25T01:44:17.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f6115e5d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6110069f40 0x7f611019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:17.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f6115e5d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6110069f40 0x7f611019e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44842/0 (socket says 172.21.15.78:44842) 2024-04-25T01:44:17.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f6115e5d700 1 -- 172.21.15.78:0/224750252 learned_addr learned my addr 172.21.15.78:0/224750252 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:17.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f611565c700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f611006a8e0 0x7f611019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:17.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f611665e700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61101082b0 0x7f61101988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:17.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f6115e5d700 1 -- 172.21.15.78:0/224750252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f611006a8e0 msgr2=0x7f611019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:17.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f6115e5d700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f611006a8e0 0x7f611019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:17.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f6115e5d700 1 -- 172.21.15.78:0/224750252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61101082b0 msgr2=0x7f61101988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:17.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f6115e5d700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61101082b0 0x7f61101988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:17.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f6115e5d700 1 -- 172.21.15.78:0/224750252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6110199000 con 0x7f6110069f40 2024-04-25T01:44:17.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f611565c700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f611006a8e0 0x7f611019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:44:17.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f6115e5d700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6110069f40 0x7f611019e6e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6100005910 tx=0x7f610000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:17.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.282+0000 7f6106ffd700 1 -- 172.21.15.78:0/224750252 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6100027070 con 0x7f6110069f40 2024-04-25T01:44:17.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.283+0000 7f61180c1700 1 -- 172.21.15.78:0/224750252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6110199290 con 0x7f6110069f40 2024-04-25T01:44:17.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.283+0000 7f6106ffd700 1 -- 172.21.15.78:0/224750252 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6100003d70 con 0x7f6110069f40 2024-04-25T01:44:17.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.283+0000 7f6106ffd700 1 -- 172.21.15.78:0/224750252 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610000f460 con 0x7f6110069f40 2024-04-25T01:44:17.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.283+0000 7f61180c1700 1 -- 172.21.15.78:0/224750252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61101a55b0 con 0x7f6110069f40 2024-04-25T01:44:17.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.283+0000 7f61180c1700 1 -- 172.21.15.78:0/224750252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f61100629c0 con 0x7f6110069f40 2024-04-25T01:44:17.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.285+0000 7f6106ffd700 1 -- 172.21.15.78:0/224750252 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f610002a070 con 0x7f6110069f40 2024-04-25T01:44:17.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.286+0000 7f6106ffd700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f60fc0a2cb0 0x7f60fc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:17.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.286+0000 7f611565c700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f60fc0a2cb0 0x7f60fc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:17.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.286+0000 7f6106ffd700 1 -- 172.21.15.78:0/224750252 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f61000d0fd0 con 0x7f6110069f40 2024-04-25T01:44:17.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.286+0000 7f6106ffd700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125] conn(0x7f60fc0ad970 0x7f60fc0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:17.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.286+0000 7f6106ffd700 1 -- 172.21.15.78:0/224750252 --> [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f610000b0b0 con 0x7f60fc0ad970 2024-04-25T01:44:17.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.286+0000 7f6106ffd700 1 -- 172.21.15.78:0/224750252 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f61000d1420 con 0x7f6110069f40 2024-04-25T01:44:17.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.286+0000 7f611665e700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125] conn(0x7f60fc0ad970 0x7f60fc0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:17.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.286+0000 7f611565c700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f60fc0a2cb0 0x7f60fc0a5170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f61100787c0 tx=0x7f610c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:17.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.287+0000 7f611665e700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125] conn(0x7f60fc0ad970 0x7f60fc0afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:17.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.288+0000 7f6106ffd700 1 -- 172.21.15.78:0/224750252 <== osd.6 v2:172.21.15.78:6818/4272451125 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f610000b0b0 con 0x7f60fc0ad970 2024-04-25T01:44:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.304+0000 7f61180c1700 1 -- 172.21.15.78:0/224750252 --> [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6110062dc0 con 0x7f60fc0ad970 2024-04-25T01:44:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.305+0000 7f6106ffd700 1 -- 172.21.15.78:0/224750252 <== osd.6 v2:172.21.15.78:6818/4272451125 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6110062dc0 con 0x7f60fc0ad970 2024-04-25T01:44:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.305+0000 7f61180c1700 1 -- 172.21.15.78:0/224750252 >> [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125] conn(0x7f60fc0ad970 msgr2=0x7f60fc0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.305+0000 7f61180c1700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125] conn(0x7f60fc0ad970 0x7f60fc0afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.305+0000 7f61180c1700 1 -- 172.21.15.78:0/224750252 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f60fc0a2cb0 msgr2=0x7f60fc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:17.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.305+0000 7f61180c1700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f60fc0a2cb0 0x7f60fc0a5170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f61100787c0 tx=0x7f610c009260 comp rx=0 tx=0).stop 2024-04-25T01:44:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.305+0000 7f61180c1700 1 -- 172.21.15.78:0/224750252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6110069f40 msgr2=0x7f611019e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.305+0000 7f61180c1700 1 --2- 172.21.15.78:0/224750252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6110069f40 0x7f611019e6e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6100005910 tx=0x7f610000bab0 comp rx=0 tx=0).stop 2024-04-25T01:44:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.305+0000 7f611665e700 1 -- 172.21.15.78:0/224750252 reap_dead start 2024-04-25T01:44:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.305+0000 7f61180c1700 1 -- 172.21.15.78:0/224750252 shutdown_connections 2024-04-25T01:44:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.305+0000 7f61180c1700 1 -- 172.21.15.78:0/224750252 >> 172.21.15.78:0/224750252 conn(0x7f61100780e0 msgr2=0x7f61100fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.306+0000 7f61180c1700 1 -- 172.21.15.78:0/224750252 shutdown_connections 2024-04-25T01:44:17.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:17.306+0000 7f61180c1700 1 -- 172.21.15.78:0/224750252 wait complete. 2024-04-25T01:44:17.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]: dispatch 2024-04-25T01:44:17.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-25T01:44:17.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-25T01:44:17.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-25T01:44:17.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-25T01:44:17.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2024-04-25T01:44:17.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-25T01:44:17.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-25T01:44:17.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-04-25T01:44:17.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:17 smithi078 ceph-mon[29469]: pgmap v320: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T01:44:17.594 INFO:teuthology.orchestra.run.smithi078.stdout:292057776163 2024-04-25T01:44:17.594 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T01:44:17.595 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:17.616 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]: dispatch 2024-04-25T01:44:17.616 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-25T01:44:17.616 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-25T01:44:17.616 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-25T01:44:17.616 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-25T01:44:17.616 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2024-04-25T01:44:17.617 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-25T01:44:17.617 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-25T01:44:17.617 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:17 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-04-25T01:44:17.617 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:17 smithi196 ceph-mon[32978]: pgmap v320: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T01:44:17.798 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]: dispatch 2024-04-25T01:44:17.798 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-25T01:44:17.798 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-25T01:44:17.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-25T01:44:17.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-25T01:44:17.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]: dispatch 2024-04-25T01:44:17.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-25T01:44:17.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-04-25T01:44:17.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-04-25T01:44:17.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:17 smithi152 ceph-mon[32465]: pgmap v320: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T01:44:18.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.157+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2127374398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4b8101af0 msgr2=0x7fe4b810dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:18.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.157+0000 7fe4bcac5700 1 --2- 172.21.15.78:0/2127374398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4b8101af0 0x7fe4b810dfe0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000b0a0 tx=0x7fe4b000b3b0 comp rx=0 tx=0).stop 2024-04-25T01:44:18.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.157+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2127374398 shutdown_connections 2024-04-25T01:44:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.157+0000 7fe4bcac5700 1 --2- 172.21.15.78:0/2127374398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4b8101af0 0x7fe4b810dfe0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.157+0000 7fe4bcac5700 1 --2- 172.21.15.78:0/2127374398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b81071a0 0x7fe4b8107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.157+0000 7fe4bcac5700 1 --2- 172.21.15.78:0/2127374398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4b8101130 0x7fe4b81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.157+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2127374398 >> 172.21.15.78:0/2127374398 conn(0x7fe4b80780e0 msgr2=0x7fe4b80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.157+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2127374398 shutdown_connections 2024-04-25T01:44:18.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.157+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2127374398 wait complete. 2024-04-25T01:44:18.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.158+0000 7fe4bcac5700 1 Processor -- start 2024-04-25T01:44:18.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.158+0000 7fe4bcac5700 1 -- start start 2024-04-25T01:44:18.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.158+0000 7fe4bcac5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4b8101130 0x7fe4b819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:18.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.158+0000 7fe4bcac5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b8101af0 0x7fe4b819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:18.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.158+0000 7fe4bcac5700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4b81071a0 0x7fe4b8198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:18.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.158+0000 7fe4bcac5700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81114e0 con 0x7fe4b81071a0 2024-04-25T01:44:18.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.158+0000 7fe4bcac5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8111360 con 0x7fe4b8101af0 2024-04-25T01:44:18.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.158+0000 7fe4bcac5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8111660 con 0x7fe4b8101130 2024-04-25T01:44:18.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.158+0000 7fe4b6d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4b81071a0 0x7fe4b8198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:18.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.158+0000 7fe4b6d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4b81071a0 0x7fe4b8198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44872/0 (socket says 172.21.15.78:44872) 2024-04-25T01:44:18.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.158+0000 7fe4b6d9d700 1 -- 172.21.15.78:0/2055785278 learned_addr learned my addr 172.21.15.78:0/2055785278 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:18.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.158+0000 7fe4b5d9b700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b8101af0 0x7fe4b819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:18.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.158+0000 7fe4b659c700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4b8101130 0x7fe4b819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.159+0000 7fe4b6d9d700 1 -- 172.21.15.78:0/2055785278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4b8101130 msgr2=0x7fe4b819e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.159+0000 7fe4b6d9d700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4b8101130 0x7fe4b819e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.159+0000 7fe4b6d9d700 1 -- 172.21.15.78:0/2055785278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b8101af0 msgr2=0x7fe4b819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.159+0000 7fe4b6d9d700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4b8101af0 0x7fe4b819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.159+0000 7fe4b6d9d700 1 -- 172.21.15.78:0/2055785278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b8198ff0 con 0x7fe4b81071a0 2024-04-25T01:44:18.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.159+0000 7fe4b659c700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4b8101130 0x7fe4b819e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:44:18.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.159+0000 7fe4b6d9d700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4b81071a0 0x7fe4b8198890 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000d010 tx=0x7fe4b000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:18.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.159+0000 7fe4a77fe700 1 -- 172.21.15.78:0/2055785278 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0021070 con 0x7fe4b81071a0 2024-04-25T01:44:18.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.159+0000 7fe4a77fe700 1 -- 172.21.15.78:0/2055785278 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b0007c80 con 0x7fe4b81071a0 2024-04-25T01:44:18.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.159+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2055785278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b8199280 con 0x7fe4b81071a0 2024-04-25T01:44:18.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.159+0000 7fe4a77fe700 1 -- 172.21.15.78:0/2055785278 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b001d450 con 0x7fe4b81071a0 2024-04-25T01:44:18.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.159+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2055785278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b81a55a0 con 0x7fe4b81071a0 2024-04-25T01:44:18.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.160+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2055785278 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe4b806a1a0 con 0x7fe4b81071a0 2024-04-25T01:44:18.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.161+0000 7fe4a77fe700 1 -- 172.21.15.78:0/2055785278 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe4b00040b0 con 0x7fe4b81071a0 2024-04-25T01:44:18.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.162+0000 7fe4a77fe700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4a00a2cb0 0x7fe4a00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:18.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.162+0000 7fe4b659c700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4a00a2cb0 0x7fe4a00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:18.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.162+0000 7fe4a77fe700 1 -- 172.21.15.78:0/2055785278 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fe4b000f020 con 0x7fe4b81071a0 2024-04-25T01:44:18.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.162+0000 7fe4a77fe700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476] conn(0x7fe4a00add90 0x7fe4a00b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:18.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.162+0000 7fe4a77fe700 1 -- 172.21.15.78:0/2055785278 --> [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4a00b0900 con 0x7fe4a00add90 2024-04-25T01:44:18.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.162+0000 7fe4a77fe700 1 -- 172.21.15.78:0/2055785278 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe4b00d1150 con 0x7fe4b81071a0 2024-04-25T01:44:18.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.162+0000 7fe4b659c700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4a00a2cb0 0x7fe4a00a5170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe4b81999d0 tx=0x7fe4a800a680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:18.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.162+0000 7fe4b5d9b700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476] conn(0x7fe4a00add90 0x7fe4a00b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:18.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.163+0000 7fe4b5d9b700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476] conn(0x7fe4a00add90 0x7fe4a00b0230 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:18.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.163+0000 7fe4a77fe700 1 -- 172.21.15.78:0/2055785278 <== osd.10 v2:172.21.15.152:6824/3363913476 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4a00b0900 con 0x7fe4a00add90 2024-04-25T01:44:18.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.181+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2055785278 --> [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe4b8102850 con 0x7fe4a00add90 2024-04-25T01:44:18.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.182+0000 7fe4a77fe700 1 -- 172.21.15.78:0/2055785278 <== osd.10 v2:172.21.15.152:6824/3363913476 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe4b8102850 con 0x7fe4a00add90 2024-04-25T01:44:18.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.182+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2055785278 >> [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476] conn(0x7fe4a00add90 msgr2=0x7fe4a00b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.182+0000 7fe4bcac5700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476] conn(0x7fe4a00add90 0x7fe4a00b0230 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.182+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2055785278 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4a00a2cb0 msgr2=0x7fe4a00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.182+0000 7fe4bcac5700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4a00a2cb0 0x7fe4a00a5170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe4b81999d0 tx=0x7fe4a800a680 comp rx=0 tx=0).stop 2024-04-25T01:44:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.182+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2055785278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4b81071a0 msgr2=0x7fe4b8198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:18.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.182+0000 7fe4bcac5700 1 --2- 172.21.15.78:0/2055785278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4b81071a0 0x7fe4b8198890 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000d010 tx=0x7fe4b000bde0 comp rx=0 tx=0).stop 2024-04-25T01:44:18.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.182+0000 7fe4b6d9d700 1 -- 172.21.15.78:0/2055785278 reap_dead start 2024-04-25T01:44:18.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.182+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2055785278 shutdown_connections 2024-04-25T01:44:18.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.182+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2055785278 >> 172.21.15.78:0/2055785278 conn(0x7fe4b80780e0 msgr2=0x7fe4b8105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:18.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.182+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2055785278 shutdown_connections 2024-04-25T01:44:18.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.182+0000 7fe4bcac5700 1 -- 172.21.15.78:0/2055785278 wait complete. 2024-04-25T01:44:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]': finished 2024-04-25T01:44:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-25T01:44:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-25T01:44:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-25T01:44:18.749 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-25T01:44:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2024-04-25T01:44:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-25T01:44:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-25T01:44:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-04-25T01:44:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:18 smithi078 ceph-mon[29469]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T01:44:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:18.750 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:18 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:18.782 INFO:teuthology.orchestra.run.smithi078.stdout:201863462963 2024-04-25T01:44:18.782 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T01:44:18.798 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]': finished 2024-04-25T01:44:18.798 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-25T01:44:18.798 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-25T01:44:18.798 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-25T01:44:18.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-25T01:44:18.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2024-04-25T01:44:18.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-25T01:44:18.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-25T01:44:18.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-04-25T01:44:18.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:18 smithi152 ceph-mon[32465]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T01:44:18.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:18.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:18.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:18.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:18 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:18.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]': finished 2024-04-25T01:44:18.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-25T01:44:18.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-25T01:44:18.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-25T01:44:18.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-25T01:44:18.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 10]}]': finished 2024-04-25T01:44:18.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-25T01:44:18.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-04-25T01:44:18.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-04-25T01:44:18.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:18 smithi196 ceph-mon[32978]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T01:44:18.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:18.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:18.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:18.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.989+0000 7fa9d6b18700 1 -- 172.21.15.78:0/1444816746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9d006b170 msgr2=0x7fa9d010f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.989+0000 7fa9d6b18700 1 --2- 172.21.15.78:0/1444816746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9d006b170 0x7fa9d010f0e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800b0a0 tx=0x7fa9c800b3b0 comp rx=0 tx=0).stop 2024-04-25T01:44:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.989+0000 7fa9d6b18700 1 -- 172.21.15.78:0/1444816746 shutdown_connections 2024-04-25T01:44:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.989+0000 7fa9d6b18700 1 --2- 172.21.15.78:0/1444816746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9d01082a0 0x7fa9d0108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.989+0000 7fa9d6b18700 1 --2- 172.21.15.78:0/1444816746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9d006b170 0x7fa9d010f0e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.989+0000 7fa9d6b18700 1 --2- 172.21.15.78:0/1444816746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9d006a7d0 0x7fa9d006ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.989+0000 7fa9d6b18700 1 -- 172.21.15.78:0/1444816746 >> 172.21.15.78:0/1444816746 conn(0x7fa9d00780a0 msgr2=0x7fa9d00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:18.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.989+0000 7fa9d6b18700 1 -- 172.21.15.78:0/1444816746 shutdown_connections 2024-04-25T01:44:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.989+0000 7fa9d6b18700 1 -- 172.21.15.78:0/1444816746 wait complete. 2024-04-25T01:44:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.990+0000 7fa9d6b18700 1 Processor -- start 2024-04-25T01:44:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.990+0000 7fa9d6b18700 1 -- start start 2024-04-25T01:44:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.990+0000 7fa9d6b18700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9d006a7d0 0x7fa9d019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.990+0000 7fa9d6b18700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9d006b170 0x7fa9d019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.990+0000 7fa9d6b18700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9d01082a0 0x7fa9d01987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.990+0000 7fa9d6b18700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d0112520 con 0x7fa9d006b170 2024-04-25T01:44:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.990+0000 7fa9d6b18700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d01123a0 con 0x7fa9d006a7d0 2024-04-25T01:44:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.990+0000 7fa9d6b18700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d01126a0 con 0x7fa9d01082a0 2024-04-25T01:44:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.990+0000 7fa9cffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9d006b170 0x7fa9d019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.990+0000 7fa9d48b4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9d006a7d0 0x7fa9d019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.990+0000 7fa9d48b4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9d006a7d0 0x7fa9d019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.78:47670/0 (socket says 172.21.15.78:47670) 2024-04-25T01:44:18.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.990+0000 7fa9d48b4700 1 -- 172.21.15.78:0/3725794111 learned_addr learned my addr 172.21.15.78:0/3725794111 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:18.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.990+0000 7fa9d50b5700 1 --2- 172.21.15.78:0/3725794111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9d01082a0 0x7fa9d01987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.991+0000 7fa9d48b4700 1 -- 172.21.15.78:0/3725794111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9d01082a0 msgr2=0x7fa9d01987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.991+0000 7fa9d48b4700 1 --2- 172.21.15.78:0/3725794111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9d01082a0 0x7fa9d01987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.991+0000 7fa9d48b4700 1 -- 172.21.15.78:0/3725794111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9d006b170 msgr2=0x7fa9d019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.991+0000 7fa9d48b4700 1 --2- 172.21.15.78:0/3725794111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9d006b170 0x7fa9d019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.991+0000 7fa9d48b4700 1 -- 172.21.15.78:0/3725794111 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d0198fc0 con 0x7fa9d006a7d0 2024-04-25T01:44:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.991+0000 7fa9cffff700 1 --2- 172.21.15.78:0/3725794111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9d006b170 0x7fa9d019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:44:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.991+0000 7fa9d48b4700 1 --2- 172.21.15.78:0/3725794111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9d006a7d0 0x7fa9d019e610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400d980 tx=0x7fa9c400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.991+0000 7fa9cdffb700 1 -- 172.21.15.78:0/3725794111 <== mon.2 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4017070 con 0x7fa9d006a7d0 2024-04-25T01:44:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.991+0000 7fa9d6b18700 1 -- 172.21.15.78:0/3725794111 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d01992b0 con 0x7fa9d006a7d0 2024-04-25T01:44:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.991+0000 7fa9cdffb700 1 -- 172.21.15.78:0/3725794111 <== mon.2 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c4005020 con 0x7fa9d006a7d0 2024-04-25T01:44:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.991+0000 7fa9cdffb700 1 -- 172.21.15.78:0/3725794111 <== mon.2 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400fa70 con 0x7fa9d006a7d0 2024-04-25T01:44:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.991+0000 7fa9d6b18700 1 -- 172.21.15.78:0/3725794111 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d0101120 con 0x7fa9d006a7d0 2024-04-25T01:44:18.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.992+0000 7fa9d6b18700 1 -- 172.21.15.78:0/3725794111 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9d00629c0 con 0x7fa9d006a7d0 2024-04-25T01:44:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.993+0000 7fa9cdffb700 1 -- 172.21.15.78:0/3725794111 <== mon.2 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa9c40051d0 con 0x7fa9d006a7d0 2024-04-25T01:44:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.994+0000 7fa9cdffb700 1 --2- 172.21.15.78:0/3725794111 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa9c00a2cb0 0x7fa9c00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:18.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.994+0000 7fa9cffff700 1 --2- 172.21.15.78:0/3725794111 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa9c00a2cb0 0x7fa9c00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.994+0000 7fa9cdffb700 1 -- 172.21.15.78:0/3725794111 <== mon.2 v2:172.21.15.152:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fa9c40cf690 con 0x7fa9d006a7d0 2024-04-25T01:44:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.994+0000 7fa9cdffb700 1 --2- 172.21.15.78:0/3725794111 >> [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265] conn(0x7fa9c00add90 0x7fa9c00b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.994+0000 7fa9cdffb700 1 -- 172.21.15.78:0/3725794111 --> [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9c4009090 con 0x7fa9c00add90 2024-04-25T01:44:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.994+0000 7fa9cdffb700 1 -- 172.21.15.78:0/3725794111 <== mon.2 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fa9c40cfa40 con 0x7fa9d006a7d0 2024-04-25T01:44:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.994+0000 7fa9d50b5700 1 --2- 172.21.15.78:0/3725794111 >> [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265] conn(0x7fa9c00add90 0x7fa9c00b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.994+0000 7fa9cffff700 1 --2- 172.21.15.78:0/3725794111 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa9c00a2cb0 0x7fa9c00a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00786a0 tx=0x7fa9bc0073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.995+0000 7fa9d50b5700 1 --2- 172.21.15.78:0/3725794111 >> [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265] conn(0x7fa9c00add90 0x7fa9c00b0230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:18.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:18.995+0000 7fa9cdffb700 1 -- 172.21.15.78:0/3725794111 <== osd.3 v2:172.21.15.78:6810/3760935265 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9c4009090 con 0x7fa9c00add90 2024-04-25T01:44:19.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:19.013+0000 7fa9d6b18700 1 -- 172.21.15.78:0/3725794111 --> [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9d0062dc0 con 0x7fa9c00add90 2024-04-25T01:44:19.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:19.013+0000 7fa9cdffb700 1 -- 172.21.15.78:0/3725794111 <== osd.3 v2:172.21.15.78:6810/3760935265 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa9d0062dc0 con 0x7fa9c00add90 2024-04-25T01:44:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:19.014+0000 7fa9d6b18700 1 -- 172.21.15.78:0/3725794111 >> [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265] conn(0x7fa9c00add90 msgr2=0x7fa9c00b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:19.014+0000 7fa9d6b18700 1 --2- 172.21.15.78:0/3725794111 >> [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265] conn(0x7fa9c00add90 0x7fa9c00b0230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:19.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:19.014+0000 7fa9d6b18700 1 -- 172.21.15.78:0/3725794111 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa9c00a2cb0 msgr2=0x7fa9c00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:19.014+0000 7fa9d6b18700 1 --2- 172.21.15.78:0/3725794111 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa9c00a2cb0 0x7fa9c00a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00786a0 tx=0x7fa9bc0073d0 comp rx=0 tx=0).stop 2024-04-25T01:44:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:19.014+0000 7fa9d6b18700 1 -- 172.21.15.78:0/3725794111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9d006a7d0 msgr2=0x7fa9d019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:19.014+0000 7fa9d6b18700 1 --2- 172.21.15.78:0/3725794111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9d006a7d0 0x7fa9d019e610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400d980 tx=0x7fa9c400dc90 comp rx=0 tx=0).stop 2024-04-25T01:44:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:19.014+0000 7fa9d50b5700 1 -- 172.21.15.78:0/3725794111 reap_dead start 2024-04-25T01:44:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:19.014+0000 7fa9d6b18700 1 -- 172.21.15.78:0/3725794111 shutdown_connections 2024-04-25T01:44:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:19.014+0000 7fa9d6b18700 1 -- 172.21.15.78:0/3725794111 >> 172.21.15.78:0/3725794111 conn(0x7fa9d00780a0 msgr2=0x7fa9d00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:19.014+0000 7fa9d6b18700 1 -- 172.21.15.78:0/3725794111 shutdown_connections 2024-04-25T01:44:19.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:19.014+0000 7fa9d6b18700 1 -- 172.21.15.78:0/3725794111 wait complete. 2024-04-25T01:44:19.730 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:19 smithi078 ceph-mon[29469]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T01:44:19.731 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:19 smithi078 ceph-mon[29469]: pgmap v323: 129 pgs: 3 peering, 126 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T01:44:19.761 INFO:teuthology.orchestra.run.smithi078.stdout:313532612639 2024-04-25T01:44:19.761 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T01:44:19.798 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:19 smithi152 ceph-mon[32465]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T01:44:19.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:19 smithi152 ceph-mon[32465]: pgmap v323: 129 pgs: 3 peering, 126 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T01:44:19.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:19 smithi196 ceph-mon[32978]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T01:44:19.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:19 smithi196 ceph-mon[32978]: pgmap v323: 129 pgs: 3 peering, 126 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T01:44:20.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.163+0000 7f796b12b700 1 -- 172.21.15.78:0/1162107211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7964101b90 msgr2=0x7f7964101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:20.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.163+0000 7f796b12b700 1 --2- 172.21.15.78:0/1162107211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7964101b90 0x7f7964101f70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f79580099c0 tx=0x7f7958009cd0 comp rx=0 tx=0).stop 2024-04-25T01:44:20.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.163+0000 7f796b12b700 1 -- 172.21.15.78:0/1162107211 shutdown_connections 2024-04-25T01:44:20.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.163+0000 7f796b12b700 1 --2- 172.21.15.78:0/1162107211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79641064b0 0x7f796410e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:20.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.163+0000 7f796b12b700 1 --2- 172.21.15.78:0/1162107211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7964101b90 0x7f7964101f70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:20.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.163+0000 7f796b12b700 1 --2- 172.21.15.78:0/1162107211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7964102540 0x7f7964105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:20.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.163+0000 7f796b12b700 1 -- 172.21.15.78:0/1162107211 >> 172.21.15.78:0/1162107211 conn(0x7f79640777a0 msgr2=0x7f79640feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:20.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.163+0000 7f796b12b700 1 -- 172.21.15.78:0/1162107211 shutdown_connections 2024-04-25T01:44:20.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.163+0000 7f796b12b700 1 -- 172.21.15.78:0/1162107211 wait complete. 2024-04-25T01:44:20.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.164+0000 7f796b12b700 1 Processor -- start 2024-04-25T01:44:20.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.164+0000 7f796b12b700 1 -- start start 2024-04-25T01:44:20.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.164+0000 7f796b12b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7964101b90 0x7f796419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:20.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.164+0000 7f796b12b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7964102540 0x7f796419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:20.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.164+0000 7f796b12b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79641064b0 0x7f796419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:20.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.164+0000 7f796b12b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7964111620 con 0x7f7964102540 2024-04-25T01:44:20.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.164+0000 7f796b12b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f79641114a0 con 0x7f79641064b0 2024-04-25T01:44:20.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.164+0000 7f796b12b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f79641117a0 con 0x7f7964101b90 2024-04-25T01:44:20.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.164+0000 7f7963fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7964102540 0x7f796419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:20.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.164+0000 7f7963fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7964102540 0x7f796419b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42732/0 (socket says 172.21.15.78:42732) 2024-04-25T01:44:20.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.164+0000 7f7963fff700 1 -- 172.21.15.78:0/1701620015 learned_addr learned my addr 172.21.15.78:0/1701620015 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:20.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.164+0000 7f79696c8700 1 --2- 172.21.15.78:0/1701620015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79641064b0 0x7f796419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:20.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.164+0000 7f7968ec7700 1 --2- 172.21.15.78:0/1701620015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7964101b90 0x7f796419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:20.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.165+0000 7f7963fff700 1 -- 172.21.15.78:0/1701620015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7964101b90 msgr2=0x7f796419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:20.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.165+0000 7f7963fff700 1 --2- 172.21.15.78:0/1701620015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7964101b90 0x7f796419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:20.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.165+0000 7f7963fff700 1 -- 172.21.15.78:0/1701620015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79641064b0 msgr2=0x7f796419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:20.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.165+0000 7f7963fff700 1 --2- 172.21.15.78:0/1701620015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79641064b0 0x7f796419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:20.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.165+0000 7f7963fff700 1 -- 172.21.15.78:0/1701620015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796419fdc0 con 0x7f7964102540 2024-04-25T01:44:20.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.165+0000 7f7963fff700 1 --2- 172.21.15.78:0/1701620015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7964102540 0x7f796419b2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f795000b500 tx=0x7f795000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:20.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.165+0000 7f7961ffb700 1 -- 172.21.15.78:0/1701620015 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950015020 con 0x7f7964102540 2024-04-25T01:44:20.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.165+0000 7f7961ffb700 1 -- 172.21.15.78:0/1701620015 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7950011d10 con 0x7f7964102540 2024-04-25T01:44:20.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.165+0000 7f796b12b700 1 -- 172.21.15.78:0/1701620015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79641a00b0 con 0x7f7964102540 2024-04-25T01:44:20.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.165+0000 7f7961ffb700 1 -- 172.21.15.78:0/1701620015 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795000f2e0 con 0x7f7964102540 2024-04-25T01:44:20.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.165+0000 7f796b12b700 1 -- 172.21.15.78:0/1701620015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79641a7930 con 0x7f7964102540 2024-04-25T01:44:20.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.166+0000 7f796b12b700 1 -- 172.21.15.78:0/1701620015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7964062980 con 0x7f7964102540 2024-04-25T01:44:20.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.168+0000 7f7961ffb700 1 -- 172.21.15.78:0/1701620015 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7950018020 con 0x7f7964102540 2024-04-25T01:44:20.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.168+0000 7f7961ffb700 1 --2- 172.21.15.78:0/1701620015 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f79540a2d30 0x7f79540a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:20.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.169+0000 7f7961ffb700 1 -- 172.21.15.78:0/1701620015 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f79500ceaf0 con 0x7f7964102540 2024-04-25T01:44:20.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.169+0000 7f7968ec7700 1 --2- 172.21.15.78:0/1701620015 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f79540a2d30 0x7f79540a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:20.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.169+0000 7f7961ffb700 1 --2- 172.21.15.78:0/1701620015 >> [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653] conn(0x7f79540ade10 0x7f79540b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:20.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.169+0000 7f7961ffb700 1 -- 172.21.15.78:0/1701620015 --> [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79540b0980 con 0x7f79540ade10 2024-04-25T01:44:20.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.169+0000 7f7961ffb700 1 -- 172.21.15.78:0/1701620015 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f79500d32c0 con 0x7f7964102540 2024-04-25T01:44:20.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.169+0000 7f79696c8700 1 --2- 172.21.15.78:0/1701620015 >> [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653] conn(0x7f79540ade10 0x7f79540b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:20.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.169+0000 7f7968ec7700 1 --2- 172.21.15.78:0/1701620015 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f79540a2d30 0x7f79540a51f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7958009750 tx=0x7f7958005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:20.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.170+0000 7f79696c8700 1 --2- 172.21.15.78:0/1701620015 >> [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653] conn(0x7f79540ade10 0x7f79540b02b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:20.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.171+0000 7f7961ffb700 1 -- 172.21.15.78:0/1701620015 <== osd.0 v2:172.21.15.78:6802/519023653 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79540b0980 con 0x7f79540ade10 2024-04-25T01:44:20.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.191+0000 7f796b12b700 1 -- 172.21.15.78:0/1701620015 --> [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7964062d80 con 0x7f79540ade10 2024-04-25T01:44:20.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.191+0000 7f7961ffb700 1 -- 172.21.15.78:0/1701620015 <== osd.0 v2:172.21.15.78:6802/519023653 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7964062d80 con 0x7f79540ade10 2024-04-25T01:44:20.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.192+0000 7f796b12b700 1 -- 172.21.15.78:0/1701620015 >> [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653] conn(0x7f79540ade10 msgr2=0x7f79540b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:20.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.192+0000 7f796b12b700 1 --2- 172.21.15.78:0/1701620015 >> [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653] conn(0x7f79540ade10 0x7f79540b02b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:20.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.192+0000 7f796b12b700 1 -- 172.21.15.78:0/1701620015 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f79540a2d30 msgr2=0x7f79540a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:20.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.192+0000 7f796b12b700 1 --2- 172.21.15.78:0/1701620015 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f79540a2d30 0x7f79540a51f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7958009750 tx=0x7f7958005fb0 comp rx=0 tx=0).stop 2024-04-25T01:44:20.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.192+0000 7f796b12b700 1 -- 172.21.15.78:0/1701620015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7964102540 msgr2=0x7f796419b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:20.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.192+0000 7f796b12b700 1 --2- 172.21.15.78:0/1701620015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7964102540 0x7f796419b2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f795000b500 tx=0x7f795000b810 comp rx=0 tx=0).stop 2024-04-25T01:44:20.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.192+0000 7f79696c8700 1 -- 172.21.15.78:0/1701620015 reap_dead start 2024-04-25T01:44:20.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.192+0000 7f796b12b700 1 -- 172.21.15.78:0/1701620015 shutdown_connections 2024-04-25T01:44:20.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.192+0000 7f796b12b700 1 -- 172.21.15.78:0/1701620015 >> 172.21.15.78:0/1701620015 conn(0x7f79640777a0 msgr2=0x7f7964104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:20.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.192+0000 7f796b12b700 1 -- 172.21.15.78:0/1701620015 shutdown_connections 2024-04-25T01:44:20.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:20.192+0000 7f796b12b700 1 -- 172.21.15.78:0/1701620015 wait complete. 2024-04-25T01:44:21.343 INFO:teuthology.orchestra.run.smithi078.stdout:107374182466 2024-04-25T01:44:21.404 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T01:44:22.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:21 smithi078 ceph-mon[29469]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-25T01:44:22.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:21 smithi152 ceph-mon[32465]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-25T01:44:22.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:21 smithi196 ceph-mon[32978]: pgmap v324: 129 pgs: 3 peering, 126 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-25T01:44:23.452 INFO:teuthology.orchestra.run.smithi078.stdout:34359738448 2024-04-25T01:44:23.452 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T01:44:23.455 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:23.504 INFO:teuthology.orchestra.run.smithi078.stdout:55834574924 2024-04-25T01:44:23.504 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T01:44:24.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:23 smithi078 ceph-mon[29469]: pgmap v325: 129 pgs: 3 peering, 126 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-25T01:44:24.034 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:24.038 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:24.039 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:24.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.057+0000 7f826c0dd700 1 -- 172.21.15.78:0/2920485171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82641071a0 msgr2=0x7f8264107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:24.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.057+0000 7f826c0dd700 1 --2- 172.21.15.78:0/2920485171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82641071a0 0x7f8264107580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8254009a10 tx=0x7f8254009d20 comp rx=0 tx=0).stop 2024-04-25T01:44:24.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.057+0000 7f826c0dd700 1 -- 172.21.15.78:0/2920485171 shutdown_connections 2024-04-25T01:44:24.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.057+0000 7f826c0dd700 1 --2- 172.21.15.78:0/2920485171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82641071a0 0x7f8264107580 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:24.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.057+0000 7f826c0dd700 1 --2- 172.21.15.78:0/2920485171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8264101af0 0x7f826410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:24.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.057+0000 7f826c0dd700 1 --2- 172.21.15.78:0/2920485171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8264101130 0x7f82641015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:24.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.057+0000 7f826c0dd700 1 -- 172.21.15.78:0/2920485171 >> 172.21.15.78:0/2920485171 conn(0x7f82640780e0 msgr2=0x7f82640784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:24.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.057+0000 7f826c0dd700 1 -- 172.21.15.78:0/2920485171 shutdown_connections 2024-04-25T01:44:24.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.057+0000 7f826c0dd700 1 -- 172.21.15.78:0/2920485171 wait complete. 2024-04-25T01:44:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.058+0000 7f826c0dd700 1 Processor -- start 2024-04-25T01:44:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.058+0000 7f826c0dd700 1 -- start start 2024-04-25T01:44:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.059+0000 7f826c0dd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8264101130 0x7f82640737d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.059+0000 7f826c0dd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8264101af0 0x7f8264073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.059+0000 7f826c0dd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82641071a0 0x7f826406c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.059+0000 7f826c0dd700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8264110320 con 0x7f8264101af0 2024-04-25T01:44:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.059+0000 7f826c0dd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f82641101a0 con 0x7f82641071a0 2024-04-25T01:44:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.059+0000 7f826c0dd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82641104a0 con 0x7f8264101130 2024-04-25T01:44:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.059+0000 7f8269678700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8264101af0 0x7f8264073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.059+0000 7f8269e79700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8264101130 0x7f82640737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.059+0000 7f8269678700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8264101af0 0x7f8264073d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42752/0 (socket says 172.21.15.78:42752) 2024-04-25T01:44:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.059+0000 7f8269678700 1 -- 172.21.15.78:0/2765128232 learned_addr learned my addr 172.21.15.78:0/2765128232 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.060+0000 7f8269678700 1 -- 172.21.15.78:0/2765128232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8264101130 msgr2=0x7f82640737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.060+0000 7f8269678700 1 --2- 172.21.15.78:0/2765128232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8264101130 0x7f82640737d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.060+0000 7f8269678700 1 -- 172.21.15.78:0/2765128232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82641071a0 msgr2=0x7f826406c860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:44:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.060+0000 7f8269678700 1 --2- 172.21.15.78:0/2765128232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82641071a0 0x7f826406c860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.060+0000 7f8269678700 1 -- 172.21.15.78:0/2765128232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826406d160 con 0x7f8264101af0 2024-04-25T01:44:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.060+0000 7f8269678700 1 --2- 172.21.15.78:0/2765128232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8264101af0 0x7f8264073d10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f826000ecf0 tx=0x7f826000c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:24.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.061+0000 7f825affd700 1 -- 172.21.15.78:0/2765128232 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826000cdc0 con 0x7f8264101af0 2024-04-25T01:44:24.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.061+0000 7f825affd700 1 -- 172.21.15.78:0/2765128232 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8260004500 con 0x7f8264101af0 2024-04-25T01:44:24.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.061+0000 7f825affd700 1 -- 172.21.15.78:0/2765128232 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260010690 con 0x7f8264101af0 2024-04-25T01:44:24.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.062+0000 7f826c0dd700 1 -- 172.21.15.78:0/2765128232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826406d450 con 0x7f8264101af0 2024-04-25T01:44:24.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.062+0000 7f826c0dd700 1 -- 172.21.15.78:0/2765128232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82640714d0 con 0x7f8264101af0 2024-04-25T01:44:24.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.063+0000 7f826c0dd700 1 -- 172.21.15.78:0/2765128232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8264100490 con 0x7f8264101af0 2024-04-25T01:44:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.066+0000 7f825affd700 1 -- 172.21.15.78:0/2765128232 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8260004020 con 0x7f8264101af0 2024-04-25T01:44:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.067+0000 7f825affd700 1 --2- 172.21.15.78:0/2765128232 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f82500a2e00 0x7f82500a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.067+0000 7f825affd700 1 -- 172.21.15.78:0/2765128232 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f82600cf950 con 0x7f8264101af0 2024-04-25T01:44:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.068+0000 7f825affd700 1 --2- 172.21.15.78:0/2765128232 >> [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251] conn(0x7f82500adee0 0x7f82500b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.068+0000 7f825affd700 1 -- 172.21.15.78:0/2765128232 --> [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8260005770 con 0x7f82500adee0 2024-04-25T01:44:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.068+0000 7f825affd700 1 -- 172.21.15.78:0/2765128232 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f82600cfda0 con 0x7f8264101af0 2024-04-25T01:44:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.068+0000 7f8269e79700 1 --2- 172.21.15.78:0/2765128232 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f82500a2e00 0x7f82500a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.068+0000 7f826a67a700 1 --2- 172.21.15.78:0/2765128232 >> [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251] conn(0x7f82500adee0 0x7f82500b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:24.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.069+0000 7f8269e79700 1 --2- 172.21.15.78:0/2765128232 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f82500a2e00 0x7f82500a52c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8254009750 tx=0x7f8254010f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:24.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.071+0000 7f826a67a700 1 --2- 172.21.15.78:0/2765128232 >> [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251] conn(0x7f82500adee0 0x7f82500b0380 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.072+0000 7f825affd700 1 -- 172.21.15.78:0/2765128232 <== osd.7 v2:172.21.15.152:6816/4200178251 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8260005770 con 0x7f82500adee0 2024-04-25T01:44:24.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.098+0000 7f826c0dd700 1 -- 172.21.15.78:0/2765128232 --> [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f82641006d0 con 0x7f82500adee0 2024-04-25T01:44:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.099+0000 7f825affd700 1 -- 172.21.15.78:0/2765128232 <== osd.7 v2:172.21.15.152:6816/4200178251 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f82641006d0 con 0x7f82500adee0 2024-04-25T01:44:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.099+0000 7f826c0dd700 1 -- 172.21.15.78:0/2765128232 >> [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251] conn(0x7f82500adee0 msgr2=0x7f82500b0380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:24.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.099+0000 7f826c0dd700 1 --2- 172.21.15.78:0/2765128232 >> [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251] conn(0x7f82500adee0 0x7f82500b0380 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:24.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.100+0000 7f826c0dd700 1 -- 172.21.15.78:0/2765128232 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f82500a2e00 msgr2=0x7f82500a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:24.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.100+0000 7f826c0dd700 1 --2- 172.21.15.78:0/2765128232 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f82500a2e00 0x7f82500a52c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8254009750 tx=0x7f8254010f90 comp rx=0 tx=0).stop 2024-04-25T01:44:24.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.100+0000 7f826c0dd700 1 -- 172.21.15.78:0/2765128232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8264101af0 msgr2=0x7f8264073d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:24.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.100+0000 7f826c0dd700 1 --2- 172.21.15.78:0/2765128232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8264101af0 0x7f8264073d10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f826000ecf0 tx=0x7f826000c5b0 comp rx=0 tx=0).stop 2024-04-25T01:44:24.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.100+0000 7f826a67a700 1 -- 172.21.15.78:0/2765128232 reap_dead start 2024-04-25T01:44:24.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.100+0000 7f826c0dd700 1 -- 172.21.15.78:0/2765128232 shutdown_connections 2024-04-25T01:44:24.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.100+0000 7f826c0dd700 1 -- 172.21.15.78:0/2765128232 >> 172.21.15.78:0/2765128232 conn(0x7f82640780e0 msgr2=0x7f82640ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:24.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.100+0000 7f826c0dd700 1 -- 172.21.15.78:0/2765128232 shutdown_connections 2024-04-25T01:44:24.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:24.100+0000 7f826c0dd700 1 -- 172.21.15.78:0/2765128232 wait complete. 2024-04-25T01:44:24.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:23 smithi152 ceph-mon[32465]: pgmap v325: 129 pgs: 3 peering, 126 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-25T01:44:24.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:23 smithi196 ceph-mon[32978]: pgmap v325: 129 pgs: 3 peering, 126 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-25T01:44:24.776 INFO:teuthology.orchestra.run.smithi078.stdout:223338299439 2024-04-25T01:44:24.777 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T01:44:24.842 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:24.850 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:24.851 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:24.851 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:24.854 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:25.976 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:25 smithi078 ceph-mon[29469]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T01:44:26.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:25 smithi152 ceph-mon[32465]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T01:44:26.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:25 smithi196 ceph-mon[32978]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 12 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T01:44:27.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.028+0000 7fe9cd9df700 1 -- 172.21.15.78:0/3603957919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c8101af0 msgr2=0x7fe9c810dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:27.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.028+0000 7fe9cd9df700 1 --2- 172.21.15.78:0/3603957919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c8101af0 0x7fe9c810dfe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000b0a0 tx=0x7fe9c000b3b0 comp rx=0 tx=0).stop 2024-04-25T01:44:27.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.028+0000 7fe9cd9df700 1 -- 172.21.15.78:0/3603957919 shutdown_connections 2024-04-25T01:44:27.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.028+0000 7fe9cd9df700 1 --2- 172.21.15.78:0/3603957919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c8101af0 0x7fe9c810dfe0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:27.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.028+0000 7fe9cd9df700 1 --2- 172.21.15.78:0/3603957919 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9c81071a0 0x7fe9c8107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:27.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.028+0000 7fe9cd9df700 1 --2- 172.21.15.78:0/3603957919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c8101130 0x7fe9c81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:27.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.028+0000 7fe9cd9df700 1 -- 172.21.15.78:0/3603957919 >> 172.21.15.78:0/3603957919 conn(0x7fe9c80780e0 msgr2=0x7fe9c80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:27.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.028+0000 7fe9cd9df700 1 -- 172.21.15.78:0/3603957919 shutdown_connections 2024-04-25T01:44:27.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.028+0000 7fe9cd9df700 1 -- 172.21.15.78:0/3603957919 wait complete. 2024-04-25T01:44:27.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.029+0000 7fe9cd9df700 1 Processor -- start 2024-04-25T01:44:27.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.029+0000 7fe9cd9df700 1 -- start start 2024-04-25T01:44:27.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.029+0000 7fe9cd9df700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c8101130 0x7fe9c8102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:27.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.029+0000 7fe9cd9df700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c8101af0 0x7fe9c8103350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:27.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.029+0000 7fe9cd9df700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9c81071a0 0x7fe9c8103890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:27.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.029+0000 7fe9cd9df700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81114c0 con 0x7fe9c8101130 2024-04-25T01:44:27.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.029+0000 7fe9cd9df700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8111340 con 0x7fe9c81071a0 2024-04-25T01:44:27.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.029+0000 7fe9cd9df700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8111640 con 0x7fe9c8101af0 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9c6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c8101130 0x7fe9c8102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9c67fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c8101af0 0x7fe9c8103350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9c6ffd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c8101130 0x7fe9c8102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42764/0 (socket says 172.21.15.78:42764) 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9c67fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c8101af0 0x7fe9c8103350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.78:50000/0 (socket says 172.21.15.78:50000) 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9c6ffd700 1 -- 172.21.15.78:0/492324875 learned_addr learned my addr 172.21.15.78:0/492324875 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9c77fe700 1 --2- 172.21.15.78:0/492324875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9c81071a0 0x7fe9c8103890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9c6ffd700 1 -- 172.21.15.78:0/492324875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c8101af0 msgr2=0x7fe9c8103350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9c6ffd700 1 --2- 172.21.15.78:0/492324875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9c8101af0 0x7fe9c8103350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9c6ffd700 1 -- 172.21.15.78:0/492324875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9c81071a0 msgr2=0x7fe9c8103890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9c6ffd700 1 --2- 172.21.15.78:0/492324875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9c81071a0 0x7fe9c8103890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9c6ffd700 1 -- 172.21.15.78:0/492324875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c81a6fd0 con 0x7fe9c8101130 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9c6ffd700 1 --2- 172.21.15.78:0/492324875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c8101130 0x7fe9c8102e10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800b530 tx=0x7fe9b800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9cc9dd700 1 -- 172.21.15.78:0/492324875 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8015020 con 0x7fe9c8101130 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9cc9dd700 1 -- 172.21.15.78:0/492324875 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9b8011d10 con 0x7fe9c8101130 2024-04-25T01:44:27.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9cd9df700 1 -- 172.21.15.78:0/492324875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c81a72c0 con 0x7fe9c8101130 2024-04-25T01:44:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9cc9dd700 1 -- 172.21.15.78:0/492324875 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b800f2e0 con 0x7fe9c8101130 2024-04-25T01:44:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.030+0000 7fe9cd9df700 1 -- 172.21.15.78:0/492324875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c81a7850 con 0x7fe9c8101130 2024-04-25T01:44:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.031+0000 7fe9cd9df700 1 -- 172.21.15.78:0/492324875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9c81051e0 con 0x7fe9c8101130 2024-04-25T01:44:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.032+0000 7fe9cc9dd700 1 -- 172.21.15.78:0/492324875 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe9b8018020 con 0x7fe9c8101130 2024-04-25T01:44:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.033+0000 7fe9cc9dd700 1 --2- 172.21.15.78:0/492324875 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe9b00a2d30 0x7fe9b00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.033+0000 7fe9c67fc700 1 --2- 172.21.15.78:0/492324875 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe9b00a2d30 0x7fe9b00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.033+0000 7fe9cc9dd700 1 -- 172.21.15.78:0/492324875 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fe9b80cf150 con 0x7fe9c8101130 2024-04-25T01:44:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.033+0000 7fe9cc9dd700 1 --2- 172.21.15.78:0/492324875 >> [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723] conn(0x7fe9b00ade10 0x7fe9b00b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.033+0000 7fe9cc9dd700 1 -- 172.21.15.78:0/492324875 --> [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9b00b0980 con 0x7fe9b00ade10 2024-04-25T01:44:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.033+0000 7fe9cc9dd700 1 -- 172.21.15.78:0/492324875 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fe9b80cf5a0 con 0x7fe9c8101130 2024-04-25T01:44:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.033+0000 7fe9c67fc700 1 --2- 172.21.15.78:0/492324875 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe9b00a2d30 0x7fe9b00a51f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8078670 tx=0x7fe9bc00b3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.033+0000 7fe9c77fe700 1 --2- 172.21.15.78:0/492324875 >> [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723] conn(0x7fe9b00ade10 0x7fe9b00b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:27.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.034+0000 7fe9c77fe700 1 --2- 172.21.15.78:0/492324875 >> [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723] conn(0x7fe9b00ade10 0x7fe9b00b02b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.035+0000 7fe9cc9dd700 1 -- 172.21.15.78:0/492324875 <== osd.2 v2:172.21.15.196:6800/2032225723 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9b00b0980 con 0x7fe9b00ade10 2024-04-25T01:44:27.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.050+0000 7fe9cd9df700 1 -- 172.21.15.78:0/492324875 --> [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe9c8102850 con 0x7fe9b00ade10 2024-04-25T01:44:27.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.051+0000 7fe9cc9dd700 1 -- 172.21.15.78:0/492324875 <== osd.2 v2:172.21.15.196:6800/2032225723 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe9c8102850 con 0x7fe9b00ade10 2024-04-25T01:44:27.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.052+0000 7fe9cd9df700 1 -- 172.21.15.78:0/492324875 >> [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723] conn(0x7fe9b00ade10 msgr2=0x7fe9b00b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:27.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.052+0000 7fe9cd9df700 1 --2- 172.21.15.78:0/492324875 >> [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723] conn(0x7fe9b00ade10 0x7fe9b00b02b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:27.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.052+0000 7fe9cd9df700 1 -- 172.21.15.78:0/492324875 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe9b00a2d30 msgr2=0x7fe9b00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:27.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.052+0000 7fe9cd9df700 1 --2- 172.21.15.78:0/492324875 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe9b00a2d30 0x7fe9b00a51f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8078670 tx=0x7fe9bc00b3c0 comp rx=0 tx=0).stop 2024-04-25T01:44:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.052+0000 7fe9cd9df700 1 -- 172.21.15.78:0/492324875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c8101130 msgr2=0x7fe9c8102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.052+0000 7fe9cd9df700 1 --2- 172.21.15.78:0/492324875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9c8101130 0x7fe9c8102e10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800b530 tx=0x7fe9b800b840 comp rx=0 tx=0).stop 2024-04-25T01:44:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.052+0000 7fe9c77fe700 1 -- 172.21.15.78:0/492324875 reap_dead start 2024-04-25T01:44:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.052+0000 7fe9cd9df700 1 -- 172.21.15.78:0/492324875 shutdown_connections 2024-04-25T01:44:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.052+0000 7fe9cd9df700 1 -- 172.21.15.78:0/492324875 >> 172.21.15.78:0/492324875 conn(0x7fe9c80780e0 msgr2=0x7fe9c8100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.052+0000 7fe9cd9df700 1 -- 172.21.15.78:0/492324875 shutdown_connections 2024-04-25T01:44:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:27.052+0000 7fe9cd9df700 1 -- 172.21.15.78:0/492324875 wait complete. 2024-04-25T01:44:28.160 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:27 smithi078 ceph-mon[29469]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s; 12 B/s, 0 objects/s recovering 2024-04-25T01:44:28.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:27 smithi152 ceph-mon[32465]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s; 12 B/s, 0 objects/s recovering 2024-04-25T01:44:28.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:27 smithi196 ceph-mon[32978]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s; 12 B/s, 0 objects/s recovering 2024-04-25T01:44:30.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:29 smithi078 ceph-mon[29469]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-04-25T01:44:30.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:29 smithi152 ceph-mon[32465]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-04-25T01:44:30.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:29 smithi196 ceph-mon[32978]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-04-25T01:44:30.841 INFO:teuthology.orchestra.run.smithi078.stdout:77309411401 2024-04-25T01:44:30.842 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T01:44:32.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:31 smithi078 ceph-mon[29469]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T01:44:32.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:31 smithi152 ceph-mon[32465]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T01:44:32.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:31 smithi196 ceph-mon[32978]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T01:44:34.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:33 smithi078 ceph-mon[29469]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T01:44:34.298 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:33 smithi152 ceph-mon[32465]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T01:44:34.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:33 smithi196 ceph-mon[32978]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T01:44:36.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:35 smithi078 ceph-mon[29469]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T01:44:36.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:35 smithi152 ceph-mon[32465]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T01:44:36.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:35 smithi196 ceph-mon[32978]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T01:44:36.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.656+0000 7f96a3313700 1 -- 172.21.15.78:0/1172124933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c101b90 msgr2=0x7f969c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:36.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.656+0000 7f96a3313700 1 --2- 172.21.15.78:0/1172124933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c101b90 0x7f969c101f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f96980099c0 tx=0x7f9698009cd0 comp rx=0 tx=0).stop 2024-04-25T01:44:36.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.656+0000 7f96a3313700 1 -- 172.21.15.78:0/1172124933 shutdown_connections 2024-04-25T01:44:36.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.656+0000 7f96a3313700 1 --2- 172.21.15.78:0/1172124933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f969c1064b0 0x7f969c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:36.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.656+0000 7f96a3313700 1 --2- 172.21.15.78:0/1172124933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c101b90 0x7f969c101f70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.656+0000 7f96a3313700 1 --2- 172.21.15.78:0/1172124933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f969c102540 0x7f969c105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.656+0000 7f96a3313700 1 -- 172.21.15.78:0/1172124933 >> 172.21.15.78:0/1172124933 conn(0x7f969c0777a0 msgr2=0x7f969c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.656+0000 7f96a3313700 1 -- 172.21.15.78:0/1172124933 shutdown_connections 2024-04-25T01:44:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.656+0000 7f96a3313700 1 -- 172.21.15.78:0/1172124933 wait complete. 2024-04-25T01:44:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.657+0000 7f96a3313700 1 Processor -- start 2024-04-25T01:44:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.657+0000 7f96a3313700 1 -- start start 2024-04-25T01:44:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.657+0000 7f96a3313700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f969c101b90 0x7f969c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.657+0000 7f96a3313700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f969c102540 0x7f969c199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.657+0000 7f96a3313700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c1064b0 0x7f969c1a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.657+0000 7f96a3313700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f969c111520 con 0x7f969c1064b0 2024-04-25T01:44:36.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.657+0000 7f96a3313700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f969c1113a0 con 0x7f969c102540 2024-04-25T01:44:36.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.657+0000 7f96a3313700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f969c1116a0 con 0x7f969c101b90 2024-04-25T01:44:36.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96a18b0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c1064b0 0x7f969c1a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:36.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96a18b0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c1064b0 0x7f969c1a49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51244/0 (socket says 172.21.15.78:51244) 2024-04-25T01:44:36.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96a18b0700 1 -- 172.21.15.78:0/2450463720 learned_addr learned my addr 172.21.15.78:0/2450463720 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:36.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96a10af700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f969c101b90 0x7f969c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:36.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96a08ae700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f969c102540 0x7f969c199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:36.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96a18b0700 1 -- 172.21.15.78:0/2450463720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f969c101b90 msgr2=0x7f969c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:36.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96a18b0700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f969c101b90 0x7f969c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:36.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96a18b0700 1 -- 172.21.15.78:0/2450463720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f969c102540 msgr2=0x7f969c199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:36.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96a18b0700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f969c102540 0x7f969c199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:36.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96a18b0700 1 -- 172.21.15.78:0/2450463720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969c1a4f00 con 0x7f969c1064b0 2024-04-25T01:44:36.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96a08ae700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f969c102540 0x7f969c199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:44:36.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96a18b0700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c1064b0 0x7f969c1a49c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9694007ee0 tx=0x7f969400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:36.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96927fc700 1 -- 172.21.15.78:0/2450463720 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969400ddd0 con 0x7f969c1064b0 2024-04-25T01:44:36.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96a3313700 1 -- 172.21.15.78:0/2450463720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969c1a5130 con 0x7f969c1064b0 2024-04-25T01:44:36.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96927fc700 1 -- 172.21.15.78:0/2450463720 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9694004020 con 0x7f969c1064b0 2024-04-25T01:44:36.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96927fc700 1 -- 172.21.15.78:0/2450463720 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694015660 con 0x7f969c1064b0 2024-04-25T01:44:36.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.658+0000 7f96a3313700 1 -- 172.21.15.78:0/2450463720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f969c1a55f0 con 0x7f969c1064b0 2024-04-25T01:44:36.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.659+0000 7f96a3313700 1 -- 172.21.15.78:0/2450463720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f969c102940 con 0x7f969c1064b0 2024-04-25T01:44:36.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.660+0000 7f96927fc700 1 -- 172.21.15.78:0/2450463720 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f96940041d0 con 0x7f969c1064b0 2024-04-25T01:44:36.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.661+0000 7f96927fc700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96880a2cb0 0x7f96880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:36.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.661+0000 7f96a10af700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96880a2cb0 0x7f96880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:36.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.661+0000 7f96927fc700 1 -- 172.21.15.78:0/2450463720 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f96940cf760 con 0x7f969c1064b0 2024-04-25T01:44:36.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.662+0000 7f96a10af700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96880a2cb0 0x7f96880a5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9698000c00 tx=0x7f9698005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:36.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.669+0000 7f96927fc700 1 -- 172.21.15.78:0/2450463720 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9694010070 con 0x7f969c1064b0 2024-04-25T01:44:36.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.873+0000 7f96a3313700 1 -- 172.21.15.78:0/2450463720 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f969c109660 con 0x7f969c1064b0 2024-04-25T01:44:36.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.874+0000 7f96927fc700 1 -- 172.21.15.78:0/2450463720 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f969409c220 con 0x7f969c1064b0 2024-04-25T01:44:36.866 INFO:teuthology.orchestra.run.smithi078.stdout:313532612642 2024-04-25T01:44:36.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.876+0000 7f96a3313700 1 -- 172.21.15.78:0/2450463720 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96880a2cb0 msgr2=0x7f96880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:36.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.876+0000 7f96a3313700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96880a2cb0 0x7f96880a5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9698000c00 tx=0x7f9698005fb0 comp rx=0 tx=0).stop 2024-04-25T01:44:36.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.876+0000 7f96a3313700 1 -- 172.21.15.78:0/2450463720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c1064b0 msgr2=0x7f969c1a49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:36.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.876+0000 7f96a3313700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c1064b0 0x7f969c1a49c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9694007ee0 tx=0x7f969400d3b0 comp rx=0 tx=0).stop 2024-04-25T01:44:36.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.876+0000 7f96a3313700 1 -- 172.21.15.78:0/2450463720 shutdown_connections 2024-04-25T01:44:36.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.876+0000 7f96a3313700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f96880a2cb0 0x7f96880a5170 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:36.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.876+0000 7f96a3313700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f969c101b90 0x7f969c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:36.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.876+0000 7f96a3313700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f969c1064b0 0x7f969c1a49c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:36.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.876+0000 7f96a3313700 1 --2- 172.21.15.78:0/2450463720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f969c102540 0x7f969c199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:36.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.876+0000 7f96a3313700 1 -- 172.21.15.78:0/2450463720 >> 172.21.15.78:0/2450463720 conn(0x7f969c0777a0 msgr2=0x7f969c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:36.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.876+0000 7f96a3313700 1 -- 172.21.15.78:0/2450463720 shutdown_connections 2024-04-25T01:44:36.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.876+0000 7f96a3313700 1 -- 172.21.15.78:0/2450463720 wait complete. 2024-04-25T01:44:37.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:36 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1515088087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T01:44:37.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:36 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2450463720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T01:44:37.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:36 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1515088087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T01:44:37.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:36 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2450463720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T01:44:37.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:36 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1515088087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T01:44:37.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:36 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2450463720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T01:44:37.781 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612639 got 313532612642 for osd.10 2024-04-25T01:44:37.781 DEBUG:teuthology.parallel:result is None 2024-04-25T01:44:38.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:37 smithi078 ceph-mon[29469]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:44:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.960+0000 7f5ca6883700 1 -- 172.21.15.78:0/3153731472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca006ac00 msgr2=0x7f5ca006afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.960+0000 7f5ca6883700 1 --2- 172.21.15.78:0/3153731472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca006ac00 0x7f5ca006afe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5c900099c0 tx=0x7f5c90009cd0 comp rx=0 tx=0).stop 2024-04-25T01:44:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.960+0000 7f5ca6883700 1 -- 172.21.15.78:0/3153731472 shutdown_connections 2024-04-25T01:44:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.960+0000 7f5ca6883700 1 --2- 172.21.15.78:0/3153731472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ca01075e0 0x7f5ca010f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.960+0000 7f5ca6883700 1 --2- 172.21.15.78:0/3153731472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca006ac00 0x7f5ca006afe0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.960+0000 7f5ca6883700 1 --2- 172.21.15.78:0/3153731472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ca006b5b0 0x7f5ca0106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.960+0000 7f5ca6883700 1 -- 172.21.15.78:0/3153731472 >> 172.21.15.78:0/3153731472 conn(0x7f5ca00771a0 msgr2=0x7f5ca00775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.960+0000 7f5ca6883700 1 -- 172.21.15.78:0/3153731472 shutdown_connections 2024-04-25T01:44:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.960+0000 7f5ca6883700 1 -- 172.21.15.78:0/3153731472 wait complete. 2024-04-25T01:44:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.960+0000 7f5ca6883700 1 Processor -- start 2024-04-25T01:44:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.960+0000 7f5ca6883700 1 -- start start 2024-04-25T01:44:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5ca6883700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca006ac00 0x7f5ca019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5ca6883700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ca006b5b0 0x7f5ca019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5ca6883700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ca01075e0 0x7f5ca0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5ca6883700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca0112570 con 0x7f5ca006ac00 2024-04-25T01:44:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5ca6883700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01123f0 con 0x7f5ca01075e0 2024-04-25T01:44:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5ca6883700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01126f0 con 0x7f5ca006b5b0 2024-04-25T01:44:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5c9ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca006ac00 0x7f5ca019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5c9ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca006ac00 0x7f5ca019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51224/0 (socket says 172.21.15.78:51224) 2024-04-25T01:44:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5c9ffff700 1 -- 172.21.15.78:0/1515088087 learned_addr learned my addr 172.21.15.78:0/1515088087 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5c9f7fe700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ca006b5b0 0x7f5ca019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5c9ffff700 1 -- 172.21.15.78:0/1515088087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ca006b5b0 msgr2=0x7f5ca019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5ca4e20700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ca01075e0 0x7f5ca0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5c9ffff700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ca006b5b0 0x7f5ca019ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5c9ffff700 1 -- 172.21.15.78:0/1515088087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ca01075e0 msgr2=0x7f5ca0198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5c9ffff700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ca01075e0 0x7f5ca0198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5c9ffff700 1 -- 172.21.15.78:0/1515088087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca0198fd0 con 0x7f5ca006ac00 2024-04-25T01:44:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5ca4e20700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ca01075e0 0x7f5ca0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:44:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5c9ffff700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca006ac00 0x7f5ca019e6b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5c900059d0 tx=0x7f5c9000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5c9d7fa700 1 -- 172.21.15.78:0/1515088087 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90027070 con 0x7f5ca006ac00 2024-04-25T01:44:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5c9d7fa700 1 -- 172.21.15.78:0/1515088087 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c90003680 con 0x7f5ca006ac00 2024-04-25T01:44:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.961+0000 7f5ca6883700 1 -- 172.21.15.78:0/1515088087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca0199260 con 0x7f5ca006ac00 2024-04-25T01:44:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.962+0000 7f5c9d7fa700 1 -- 172.21.15.78:0/1515088087 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90017650 con 0x7f5ca006ac00 2024-04-25T01:44:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.962+0000 7f5ca6883700 1 -- 172.21.15.78:0/1515088087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca01a5580 con 0x7f5ca006ac00 2024-04-25T01:44:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.963+0000 7f5ca6883700 1 -- 172.21.15.78:0/1515088087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca010a790 con 0x7f5ca006ac00 2024-04-25T01:44:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.964+0000 7f5c9d7fa700 1 -- 172.21.15.78:0/1515088087 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5c9002a070 con 0x7f5ca006ac00 2024-04-25T01:44:38.093 INFO:teuthology.orchestra.run.smithi078.stdout:347892351005 2024-04-25T01:44:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.964+0000 7f5c9d7fa700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5c880a2cb0 0x7f5c880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.964+0000 7f5c9f7fe700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5c880a2cb0 0x7f5c880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.964+0000 7f5c9d7fa700 1 -- 172.21.15.78:0/1515088087 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f5c900d01e0 con 0x7f5ca006ac00 2024-04-25T01:44:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.965+0000 7f5c9f7fe700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5c880a2cb0 0x7f5c880a5170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5c94005950 tx=0x7f5c940058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:35.969+0000 7f5c9d7fa700 1 -- 172.21.15.78:0/1515088087 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5c90013070 con 0x7f5ca006ac00 2024-04-25T01:44:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.203+0000 7f5ca6883700 1 -- 172.21.15.78:0/1515088087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f5ca0199da0 con 0x7f5ca006ac00 2024-04-25T01:44:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.203+0000 7f5c9d7fa700 1 -- 172.21.15.78:0/1515088087 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5c9009cd30 con 0x7f5ca006ac00 2024-04-25T01:44:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.205+0000 7f5ca6883700 1 -- 172.21.15.78:0/1515088087 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5c880a2cb0 msgr2=0x7f5c880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.205+0000 7f5ca6883700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5c880a2cb0 0x7f5c880a5170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5c94005950 tx=0x7f5c940058e0 comp rx=0 tx=0).stop 2024-04-25T01:44:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.205+0000 7f5ca6883700 1 -- 172.21.15.78:0/1515088087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca006ac00 msgr2=0x7f5ca019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.205+0000 7f5ca6883700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca006ac00 0x7f5ca019e6b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5c900059d0 tx=0x7f5c9000fab0 comp rx=0 tx=0).stop 2024-04-25T01:44:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.205+0000 7f5ca6883700 1 -- 172.21.15.78:0/1515088087 shutdown_connections 2024-04-25T01:44:38.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.205+0000 7f5ca6883700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5c880a2cb0 0x7f5c880a5170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:38.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.205+0000 7f5ca6883700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ca01075e0 0x7f5ca0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:38.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.205+0000 7f5ca6883700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ca006ac00 0x7f5ca019e6b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:38.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.205+0000 7f5ca6883700 1 --2- 172.21.15.78:0/1515088087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ca006b5b0 0x7f5ca019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:38.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.205+0000 7f5ca6883700 1 -- 172.21.15.78:0/1515088087 >> 172.21.15.78:0/1515088087 conn(0x7f5ca00771a0 msgr2=0x7f5ca0108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:38.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.205+0000 7f5ca6883700 1 -- 172.21.15.78:0/1515088087 shutdown_connections 2024-04-25T01:44:38.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:36.205+0000 7f5ca6883700 1 -- 172.21.15.78:0/1515088087 wait complete. 2024-04-25T01:44:38.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:37 smithi152 ceph-mon[32465]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:44:38.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:37 smithi196 ceph-mon[32978]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:44:38.654 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351001 got 347892351005 for osd.11 2024-04-25T01:44:38.655 DEBUG:teuthology.parallel:result is None 2024-04-25T01:44:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.908+0000 7f1f268c0700 1 -- 172.21.15.78:0/774254161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f201071e0 msgr2=0x7f1f201075c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.908+0000 7f1f268c0700 1 --2- 172.21.15.78:0/774254161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f201071e0 0x7f1f201075c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009990 tx=0x7f1f10009ca0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.908+0000 7f1f268c0700 1 -- 172.21.15.78:0/774254161 shutdown_connections 2024-04-25T01:44:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.908+0000 7f1f268c0700 1 --2- 172.21.15.78:0/774254161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f20101b20 0x7f1f2010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.908+0000 7f1f268c0700 1 --2- 172.21.15.78:0/774254161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f201071e0 0x7f1f201075c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.908+0000 7f1f268c0700 1 --2- 172.21.15.78:0/774254161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f20101160 0x7f1f201015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.908+0000 7f1f268c0700 1 -- 172.21.15.78:0/774254161 >> 172.21.15.78:0/774254161 conn(0x7f1f200771a0 msgr2=0x7f1f200775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.908+0000 7f1f268c0700 1 -- 172.21.15.78:0/774254161 shutdown_connections 2024-04-25T01:44:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.908+0000 7f1f268c0700 1 -- 172.21.15.78:0/774254161 wait complete. 2024-04-25T01:44:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.909+0000 7f1f268c0700 1 Processor -- start 2024-04-25T01:44:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.909+0000 7f1f268c0700 1 -- start start 2024-04-25T01:44:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.909+0000 7f1f268c0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f20101160 0x7f1f201a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.909+0000 7f1f268c0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f20101b20 0x7f1f201a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.909+0000 7f1f268c0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f201071e0 0x7f1f2019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.909+0000 7f1f268c0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f20111480 con 0x7f1f20101160 2024-04-25T01:44:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.909+0000 7f1f268c0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1f20111300 con 0x7f1f201071e0 2024-04-25T01:44:39.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.909+0000 7f1f268c0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f20111600 con 0x7f1f20101b20 2024-04-25T01:44:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f1ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f20101160 0x7f1f201a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f1ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f20101160 0x7f1f201a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51260/0 (socket says 172.21.15.78:51260) 2024-04-25T01:44:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f1ffff700 1 -- 172.21.15.78:0/2326099577 learned_addr learned my addr 172.21.15.78:0/2326099577 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f24e5d700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f201071e0 0x7f1f2019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f1f7fe700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f20101b20 0x7f1f201a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f1ffff700 1 -- 172.21.15.78:0/2326099577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f20101b20 msgr2=0x7f1f201a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f1ffff700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f20101b20 0x7f1f201a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f1ffff700 1 -- 172.21.15.78:0/2326099577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f201071e0 msgr2=0x7f1f2019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f1ffff700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f201071e0 0x7f1f2019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f1ffff700 1 -- 172.21.15.78:0/2326099577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2019b1f0 con 0x7f1f20101160 2024-04-25T01:44:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f1f7fe700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f20101b20 0x7f1f201a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:44:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f1ffff700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f20101160 0x7f1f201a08d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1f10000c00 tx=0x7f1f10003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f1d7fa700 1 -- 172.21.15.78:0/2326099577 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10027070 con 0x7f1f20101160 2024-04-25T01:44:39.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f268c0700 1 -- 172.21.15.78:0/2326099577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2019b480 con 0x7f1f20101160 2024-04-25T01:44:39.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f1d7fa700 1 -- 172.21.15.78:0/2326099577 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f10004110 con 0x7f1f20101160 2024-04-25T01:44:39.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f1d7fa700 1 -- 172.21.15.78:0/2326099577 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1000f460 con 0x7f1f20101160 2024-04-25T01:44:39.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.910+0000 7f1f268c0700 1 -- 172.21.15.78:0/2326099577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f201a77a0 con 0x7f1f20101160 2024-04-25T01:44:39.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.911+0000 7f1f268c0700 1 -- 172.21.15.78:0/2326099577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f20102850 con 0x7f1f20101160 2024-04-25T01:44:39.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.912+0000 7f1f1d7fa700 1 -- 172.21.15.78:0/2326099577 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1f1002a070 con 0x7f1f20101160 2024-04-25T01:44:39.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.913+0000 7f1f1d7fa700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1f080a2cb0 0x7f1f080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:39.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.913+0000 7f1f1f7fe700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1f080a2cb0 0x7f1f080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:39.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.913+0000 7f1f1d7fa700 1 -- 172.21.15.78:0/2326099577 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f1f100d08b0 con 0x7f1f20101160 2024-04-25T01:44:39.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.914+0000 7f1f1f7fe700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1f080a2cb0 0x7f1f080a5170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1f201010e0 tx=0x7f1f14008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:39.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:38.918+0000 7f1f1d7fa700 1 -- 172.21.15.78:0/2326099577 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1f10031070 con 0x7f1f20101160 2024-04-25T01:44:39.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.120+0000 7f1f268c0700 1 -- 172.21.15.78:0/2326099577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1f201a7970 con 0x7f1f20101160 2024-04-25T01:44:39.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.121+0000 7f1f1d7fa700 1 -- 172.21.15.78:0/2326099577 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1f10013070 con 0x7f1f20101160 2024-04-25T01:44:39.113 INFO:teuthology.orchestra.run.smithi078.stdout:34359738452 2024-04-25T01:44:39.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.123+0000 7f1f268c0700 1 -- 172.21.15.78:0/2326099577 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1f080a2cb0 msgr2=0x7f1f080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:39.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.123+0000 7f1f268c0700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1f080a2cb0 0x7f1f080a5170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1f201010e0 tx=0x7f1f14008040 comp rx=0 tx=0).stop 2024-04-25T01:44:39.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.123+0000 7f1f268c0700 1 -- 172.21.15.78:0/2326099577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f20101160 msgr2=0x7f1f201a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:39.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.123+0000 7f1f268c0700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f20101160 0x7f1f201a08d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1f10000c00 tx=0x7f1f10003680 comp rx=0 tx=0).stop 2024-04-25T01:44:39.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.123+0000 7f1f268c0700 1 -- 172.21.15.78:0/2326099577 shutdown_connections 2024-04-25T01:44:39.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.123+0000 7f1f268c0700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1f080a2cb0 0x7f1f080a5170 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.123+0000 7f1f268c0700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f201071e0 0x7f1f2019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.123+0000 7f1f268c0700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f20101160 0x7f1f201a08d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.123+0000 7f1f268c0700 1 --2- 172.21.15.78:0/2326099577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f20101b20 0x7f1f201a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.123+0000 7f1f268c0700 1 -- 172.21.15.78:0/2326099577 >> 172.21.15.78:0/2326099577 conn(0x7f1f200771a0 msgr2=0x7f1f2006a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:39.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.123+0000 7f1f268c0700 1 -- 172.21.15.78:0/2326099577 shutdown_connections 2024-04-25T01:44:39.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.123+0000 7f1f268c0700 1 -- 172.21.15.78:0/2326099577 wait complete. 2024-04-25T01:44:39.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.241+0000 7fc67c97f700 1 -- 172.21.15.78:0/2558542693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc678101270 msgr2=0x7fc67810d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:39.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.241+0000 7fc67c97f700 1 --2- 172.21.15.78:0/2558542693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc678101270 0x7fc67810d780 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc67000b0a0 tx=0x7fc67000b3b0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.242+0000 7fc67c97f700 1 -- 172.21.15.78:0/2558542693 shutdown_connections 2024-04-25T01:44:39.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.242+0000 7fc67c97f700 1 --2- 172.21.15.78:0/2558542693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc678101270 0x7fc67810d780 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.242+0000 7fc67c97f700 1 --2- 172.21.15.78:0/2558542693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6781068b0 0x7fc678106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.242+0000 7fc67c97f700 1 --2- 172.21.15.78:0/2558542693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6781008b0 0x7fc678100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.242+0000 7fc67c97f700 1 -- 172.21.15.78:0/2558542693 >> 172.21.15.78:0/2558542693 conn(0x7fc678077730 msgr2=0x7fc6780feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.242+0000 7fc67c97f700 1 -- 172.21.15.78:0/2558542693 shutdown_connections 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.242+0000 7fc67c97f700 1 -- 172.21.15.78:0/2558542693 wait complete. 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.242+0000 7fc67c97f700 1 Processor -- start 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.242+0000 7fc67c97f700 1 -- start start 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc67c97f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6781008b0 0x7fc67819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc67c97f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc678101270 0x7fc67819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc67c97f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6781068b0 0x7fc6781976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc67c97f700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc678110c40 con 0x7fc6781068b0 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc67c97f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc678110ac0 con 0x7fc678101270 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc67c97f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc678110dc0 con 0x7fc6781008b0 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc676d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6781068b0 0x7fc6781976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:39.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc676d9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6781068b0 0x7fc6781976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51286/0 (socket says 172.21.15.78:51286) 2024-04-25T01:44:39.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc676d9d700 1 -- 172.21.15.78:0/4062954165 learned_addr learned my addr 172.21.15.78:0/4062954165 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:39.469 INFO:teuthology.orchestra.run.smithi078.stdout:55834574928 2024-04-25T01:44:39.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc675d9b700 1 --2- 172.21.15.78:0/4062954165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc678101270 0x7fc67819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:39.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc67659c700 1 --2- 172.21.15.78:0/4062954165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6781008b0 0x7fc67819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:39.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc676d9d700 1 -- 172.21.15.78:0/4062954165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6781008b0 msgr2=0x7fc67819d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:39.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc676d9d700 1 --2- 172.21.15.78:0/4062954165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6781008b0 0x7fc67819d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc676d9d700 1 -- 172.21.15.78:0/4062954165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc678101270 msgr2=0x7fc67819db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:39.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc676d9d700 1 --2- 172.21.15.78:0/4062954165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc678101270 0x7fc67819db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc676d9d700 1 -- 172.21.15.78:0/4062954165 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc678197fc0 con 0x7fc6781068b0 2024-04-25T01:44:39.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc676d9d700 1 --2- 172.21.15.78:0/4062954165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6781068b0 0x7fc6781976e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc6700062a0 tx=0x7fc670004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:39.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.243+0000 7fc6677fe700 1 -- 172.21.15.78:0/4062954165 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc670021070 con 0x7fc6781068b0 2024-04-25T01:44:39.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.244+0000 7fc6677fe700 1 -- 172.21.15.78:0/4062954165 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc670007c00 con 0x7fc6781068b0 2024-04-25T01:44:39.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.244+0000 7fc67c97f700 1 -- 172.21.15.78:0/4062954165 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc678198250 con 0x7fc6781068b0 2024-04-25T01:44:39.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.244+0000 7fc6677fe700 1 -- 172.21.15.78:0/4062954165 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67001d450 con 0x7fc6781068b0 2024-04-25T01:44:39.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.244+0000 7fc67c97f700 1 -- 172.21.15.78:0/4062954165 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6781a4590 con 0x7fc6781068b0 2024-04-25T01:44:39.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.245+0000 7fc67c97f700 1 -- 172.21.15.78:0/4062954165 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6781985c0 con 0x7fc6781068b0 2024-04-25T01:44:39.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.246+0000 7fc6677fe700 1 -- 172.21.15.78:0/4062954165 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc670004020 con 0x7fc6781068b0 2024-04-25T01:44:39.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.246+0000 7fc6677fe700 1 --2- 172.21.15.78:0/4062954165 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc6600a2cb0 0x7fc6600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:39.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.246+0000 7fc67659c700 1 --2- 172.21.15.78:0/4062954165 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc6600a2cb0 0x7fc6600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:39.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.246+0000 7fc6677fe700 1 -- 172.21.15.78:0/4062954165 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fc67000f020 con 0x7fc6781068b0 2024-04-25T01:44:39.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.247+0000 7fc67659c700 1 --2- 172.21.15.78:0/4062954165 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc6600a2cb0 0x7fc6600a5170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc668005950 tx=0x7fc6680058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.252+0000 7fc6677fe700 1 -- 172.21.15.78:0/4062954165 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc6781985c0 con 0x7fc6781068b0 2024-04-25T01:44:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.451+0000 7fc67c97f700 1 -- 172.21.15.78:0/4062954165 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc6781a4760 con 0x7fc6781068b0 2024-04-25T01:44:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.452+0000 7fc6677fe700 1 -- 172.21.15.78:0/4062954165 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc67002a020 con 0x7fc6781068b0 2024-04-25T01:44:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.454+0000 7fc67c97f700 1 -- 172.21.15.78:0/4062954165 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc6600a2cb0 msgr2=0x7fc6600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.454+0000 7fc67c97f700 1 --2- 172.21.15.78:0/4062954165 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc6600a2cb0 0x7fc6600a5170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc668005950 tx=0x7fc6680058e0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.454+0000 7fc67c97f700 1 -- 172.21.15.78:0/4062954165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6781068b0 msgr2=0x7fc6781976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.454+0000 7fc67c97f700 1 --2- 172.21.15.78:0/4062954165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6781068b0 0x7fc6781976e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc6700062a0 tx=0x7fc670004d70 comp rx=0 tx=0).stop 2024-04-25T01:44:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.454+0000 7fc67c97f700 1 -- 172.21.15.78:0/4062954165 shutdown_connections 2024-04-25T01:44:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.454+0000 7fc67c97f700 1 --2- 172.21.15.78:0/4062954165 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc6600a2cb0 0x7fc6600a5170 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.454+0000 7fc67c97f700 1 --2- 172.21.15.78:0/4062954165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6781008b0 0x7fc67819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.454+0000 7fc67c97f700 1 --2- 172.21.15.78:0/4062954165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6781068b0 0x7fc6781976e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.454+0000 7fc67c97f700 1 --2- 172.21.15.78:0/4062954165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc678101270 0x7fc67819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.454+0000 7fc67c97f700 1 -- 172.21.15.78:0/4062954165 >> 172.21.15.78:0/4062954165 conn(0x7fc678077730 msgr2=0x7fc6781023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:39.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.454+0000 7fc67c97f700 1 -- 172.21.15.78:0/4062954165 shutdown_connections 2024-04-25T01:44:39.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:39.454+0000 7fc67c97f700 1 -- 172.21.15.78:0/4062954165 wait complete. 2024-04-25T01:44:40.072 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:39 smithi078 ceph-mon[29469]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:44:40.072 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:39 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2326099577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T01:44:40.072 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:39 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/4062954165' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T01:44:40.226 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738448 got 34359738452 for osd.0 2024-04-25T01:44:40.226 DEBUG:teuthology.parallel:result is None 2024-04-25T01:44:40.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:39 smithi152 ceph-mon[32465]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:44:40.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:39 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2326099577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T01:44:40.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:39 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/4062954165' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T01:44:40.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:39 smithi196 ceph-mon[32978]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:44:40.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:39 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2326099577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T01:44:40.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:39 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/4062954165' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T01:44:41.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.041+0000 7fea1bea4700 1 -- 172.21.15.78:0/3172121368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea141075a0 msgr2=0x7fea1410f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:41.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.041+0000 7fea1bea4700 1 --2- 172.21.15.78:0/3172121368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea141075a0 0x7fea1410f170 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009990 tx=0x7fea0c009ca0 comp rx=0 tx=0).stop 2024-04-25T01:44:41.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.041+0000 7fea1bea4700 1 -- 172.21.15.78:0/3172121368 shutdown_connections 2024-04-25T01:44:41.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.041+0000 7fea1bea4700 1 --2- 172.21.15.78:0/3172121368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea141075a0 0x7fea1410f170 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:41.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.041+0000 7fea1bea4700 1 --2- 172.21.15.78:0/3172121368 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea141024e0 0x7fea141028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:41.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.041+0000 7fea1bea4700 1 --2- 172.21.15.78:0/3172121368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea14102e90 0x7fea14106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:41.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.041+0000 7fea1bea4700 1 -- 172.21.15.78:0/3172121368 >> 172.21.15.78:0/3172121368 conn(0x7fea140793e0 msgr2=0x7fea140797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:41.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.041+0000 7fea1bea4700 1 -- 172.21.15.78:0/3172121368 shutdown_connections 2024-04-25T01:44:41.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.041+0000 7fea1bea4700 1 -- 172.21.15.78:0/3172121368 wait complete. 2024-04-25T01:44:41.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.041+0000 7fea1bea4700 1 Processor -- start 2024-04-25T01:44:41.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.041+0000 7fea1bea4700 1 -- start start 2024-04-25T01:44:41.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1bea4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea141024e0 0x7fea14074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1bea4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea14102e90 0x7fea14074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1bea4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea141075a0 0x7fea14074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1bea4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea14112600 con 0x7fea14102e90 2024-04-25T01:44:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1bea4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fea14112480 con 0x7fea141075a0 2024-04-25T01:44:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1bea4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea14112780 con 0x7fea141024e0 2024-04-25T01:44:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1943f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea14102e90 0x7fea14074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1943f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea14102e90 0x7fea14074a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35548/0 (socket says 172.21.15.78:35548) 2024-04-25T01:44:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1943f700 1 -- 172.21.15.78:0/1932599002 learned_addr learned my addr 172.21.15.78:0/1932599002 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1a441700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea141075a0 0x7fea14074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea19c40700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea141024e0 0x7fea14074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1943f700 1 -- 172.21.15.78:0/1932599002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea141024e0 msgr2=0x7fea14074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1943f700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea141024e0 0x7fea14074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:41.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1943f700 1 -- 172.21.15.78:0/1932599002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea141075a0 msgr2=0x7fea14074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:41.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1943f700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea141075a0 0x7fea14074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:41.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1943f700 1 -- 172.21.15.78:0/1932599002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea141a6f40 con 0x7fea14102e90 2024-04-25T01:44:41.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea19c40700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea141024e0 0x7fea14074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:44:41.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1943f700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea14102e90 0x7fea14074a70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fea0400b500 tx=0x7fea0400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:41.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea0affd700 1 -- 172.21.15.78:0/1932599002 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04015020 con 0x7fea14102e90 2024-04-25T01:44:41.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1bea4700 1 -- 172.21.15.78:0/1932599002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea141a7230 con 0x7fea14102e90 2024-04-25T01:44:41.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea1bea4700 1 -- 172.21.15.78:0/1932599002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea141a77c0 con 0x7fea14102e90 2024-04-25T01:44:41.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea0affd700 1 -- 172.21.15.78:0/1932599002 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea04011d10 con 0x7fea14102e90 2024-04-25T01:44:41.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.042+0000 7fea0affd700 1 -- 172.21.15.78:0/1932599002 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0400f2e0 con 0x7fea14102e90 2024-04-25T01:44:41.967 INFO:teuthology.orchestra.run.smithi078.stdout:292057776168 2024-04-25T01:44:41.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.043+0000 7fea1bea4700 1 -- 172.21.15.78:0/1932599002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f8005330 con 0x7fea14102e90 2024-04-25T01:44:41.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.044+0000 7fea0affd700 1 -- 172.21.15.78:0/1932599002 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fea04018020 con 0x7fea14102e90 2024-04-25T01:44:41.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.045+0000 7fea0affd700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea000a2cb0 0x7fea000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:41.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.045+0000 7fea19c40700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea000a2cb0 0x7fea000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:41.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.045+0000 7fea0affd700 1 -- 172.21.15.78:0/1932599002 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fea040cef00 con 0x7fea14102e90 2024-04-25T01:44:41.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.045+0000 7fea19c40700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea000a2cb0 0x7fea000a5170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fea100060b0 tx=0x7fea100076a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:41.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.049+0000 7fea0affd700 1 -- 172.21.15.78:0/1932599002 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fea0401f020 con 0x7fea14102e90 2024-04-25T01:44:41.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.251+0000 7fea1bea4700 1 -- 172.21.15.78:0/1932599002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fe9f80058f0 con 0x7fea14102e90 2024-04-25T01:44:41.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.251+0000 7fea0affd700 1 -- 172.21.15.78:0/1932599002 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fea0409bad0 con 0x7fea14102e90 2024-04-25T01:44:41.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.253+0000 7fea1bea4700 1 -- 172.21.15.78:0/1932599002 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea000a2cb0 msgr2=0x7fea000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.253+0000 7fea1bea4700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea000a2cb0 0x7fea000a5170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fea100060b0 tx=0x7fea100076a0 comp rx=0 tx=0).stop 2024-04-25T01:44:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.254+0000 7fea1bea4700 1 -- 172.21.15.78:0/1932599002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea14102e90 msgr2=0x7fea14074a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.254+0000 7fea1bea4700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea14102e90 0x7fea14074a70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fea0400b500 tx=0x7fea0400b8c0 comp rx=0 tx=0).stop 2024-04-25T01:44:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.254+0000 7fea1bea4700 1 -- 172.21.15.78:0/1932599002 shutdown_connections 2024-04-25T01:44:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.254+0000 7fea1bea4700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fea000a2cb0 0x7fea000a5170 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.254+0000 7fea1bea4700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea141024e0 0x7fea14074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.254+0000 7fea1bea4700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea141075a0 0x7fea14074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.254+0000 7fea1bea4700 1 --2- 172.21.15.78:0/1932599002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea14102e90 0x7fea14074a70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.254+0000 7fea1bea4700 1 -- 172.21.15.78:0/1932599002 >> 172.21.15.78:0/1932599002 conn(0x7fea140793e0 msgr2=0x7fea14100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.254+0000 7fea1bea4700 1 -- 172.21.15.78:0/1932599002 shutdown_connections 2024-04-25T01:44:41.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:41.254+0000 7fea1bea4700 1 -- 172.21.15.78:0/1932599002 wait complete. 2024-04-25T01:44:42.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:41 smithi078 ceph-mon[29469]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:44:42.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:41 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1932599002' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T01:44:42.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:41 smithi152 ceph-mon[32465]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:44:42.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:41 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1932599002' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T01:44:42.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:41 smithi196 ceph-mon[32978]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:44:42.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:41 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1932599002' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T01:44:42.679 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776163 got 292057776168 for osd.9 2024-04-25T01:44:42.679 DEBUG:teuthology.parallel:result is None 2024-04-25T01:44:42.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.879+0000 7f5aad89b700 1 -- 172.21.15.78:0/817396377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aa810ec30 msgr2=0x7f5aa8111020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.879+0000 7f5aad89b700 1 --2- 172.21.15.78:0/817396377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aa810ec30 0x7f5aa8111020 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000b0a0 tx=0x7f5aa000b3b0 comp rx=0 tx=0).stop 2024-04-25T01:44:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.879+0000 7f5aad89b700 1 -- 172.21.15.78:0/817396377 shutdown_connections 2024-04-25T01:44:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.879+0000 7f5aad89b700 1 --2- 172.21.15.78:0/817396377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5aa8074460 0x7f5aa8074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.879+0000 7f5aad89b700 1 --2- 172.21.15.78:0/817396377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aa810ec30 0x7f5aa8111020 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.879+0000 7f5aad89b700 1 --2- 172.21.15.78:0/817396377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5aa8074d80 0x7f5aa810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.879+0000 7f5aad89b700 1 -- 172.21.15.78:0/817396377 >> 172.21.15.78:0/817396377 conn(0x7f5aa8079390 msgr2=0x7f5aa80797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.879+0000 7f5aad89b700 1 -- 172.21.15.78:0/817396377 shutdown_connections 2024-04-25T01:44:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.879+0000 7f5aad89b700 1 -- 172.21.15.78:0/817396377 wait complete. 2024-04-25T01:44:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.879+0000 7f5aad89b700 1 Processor -- start 2024-04-25T01:44:42.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.879+0000 7f5aad89b700 1 -- start start 2024-04-25T01:44:42.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5aad89b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5aa8074460 0x7f5aa81a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5aad89b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aa8074d80 0x7f5aa81a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5aad89b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5aa810ec30 0x7f5aa819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5aad89b700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa8114700 con 0x7f5aa8074d80 2024-04-25T01:44:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5aad89b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa8114580 con 0x7f5aa8074460 2024-04-25T01:44:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5aad89b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa8114880 con 0x7f5aa810ec30 2024-04-25T01:44:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5a9e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aa8074d80 0x7f5aa81a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5a9e7fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aa8074d80 0x7f5aa81a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35574/0 (socket says 172.21.15.78:35574) 2024-04-25T01:44:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5a9e7fc700 1 -- 172.21.15.78:0/3156925552 learned_addr learned my addr 172.21.15.78:0/3156925552 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5aa6ffd700 1 --2- 172.21.15.78:0/3156925552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5aa8074460 0x7f5aa81a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5aa77fe700 1 --2- 172.21.15.78:0/3156925552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5aa810ec30 0x7f5aa819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5a9e7fc700 1 -- 172.21.15.78:0/3156925552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5aa810ec30 msgr2=0x7f5aa819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5a9e7fc700 1 --2- 172.21.15.78:0/3156925552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5aa810ec30 0x7f5aa819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:42.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5a9e7fc700 1 -- 172.21.15.78:0/3156925552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5aa8074460 msgr2=0x7f5aa81a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:42.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5a9e7fc700 1 --2- 172.21.15.78:0/3156925552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5aa8074460 0x7f5aa81a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:42.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5a9e7fc700 1 -- 172.21.15.78:0/3156925552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa819b140 con 0x7f5aa8074d80 2024-04-25T01:44:42.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.880+0000 7f5a9e7fc700 1 --2- 172.21.15.78:0/3156925552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aa8074d80 0x7f5aa81a0d10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000e9a0 tx=0x7f5a9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:42.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.881+0000 7f5aac899700 1 -- 172.21.15.78:0/3156925552 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000ca10 con 0x7f5aa8074d80 2024-04-25T01:44:42.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.881+0000 7f5aad89b700 1 -- 172.21.15.78:0/3156925552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa819b430 con 0x7f5aa8074d80 2024-04-25T01:44:42.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.881+0000 7f5aac899700 1 -- 172.21.15.78:0/3156925552 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a9000cbb0 con 0x7f5aa8074d80 2024-04-25T01:44:42.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.881+0000 7f5aac899700 1 -- 172.21.15.78:0/3156925552 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90018680 con 0x7f5aa8074d80 2024-04-25T01:44:42.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.881+0000 7f5aad89b700 1 -- 172.21.15.78:0/3156925552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa81a76a0 con 0x7f5aa8074d80 2024-04-25T01:44:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.882+0000 7f5aad89b700 1 -- 172.21.15.78:0/3156925552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa8068f20 con 0x7f5aa8074d80 2024-04-25T01:44:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.883+0000 7f5aac899700 1 -- 172.21.15.78:0/3156925552 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5a90010430 con 0x7f5aa8074d80 2024-04-25T01:44:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.883+0000 7f5aac899700 1 --2- 172.21.15.78:0/3156925552 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5a940a2cb0 0x7f5a940a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.883+0000 7f5aa6ffd700 1 --2- 172.21.15.78:0/3156925552 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5a940a2cb0 0x7f5a940a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.884+0000 7f5aac899700 1 -- 172.21.15.78:0/3156925552 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f5a900cf6c0 con 0x7f5aa8074d80 2024-04-25T01:44:42.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.884+0000 7f5aa6ffd700 1 --2- 172.21.15.78:0/3156925552 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5a940a2cb0 0x7f5a940a5170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8079a90 tx=0x7f5a9800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:42.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:42.888+0000 7f5aac899700 1 -- 172.21.15.78:0/3156925552 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5a9009c180 con 0x7f5aa8074d80 2024-04-25T01:44:43.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.090+0000 7f5aad89b700 1 -- 172.21.15.78:0/3156925552 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5aa802db30 con 0x7f5aa8074d80 2024-04-25T01:44:43.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.091+0000 7f5aac899700 1 -- 172.21.15.78:0/3156925552 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5a9009ba80 con 0x7f5aa8074d80 2024-04-25T01:44:43.083 INFO:teuthology.orchestra.run.smithi078.stdout:107374182470 2024-04-25T01:44:43.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.093+0000 7f5aad89b700 1 -- 172.21.15.78:0/3156925552 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5a940a2cb0 msgr2=0x7f5a940a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:43.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.093+0000 7f5aad89b700 1 --2- 172.21.15.78:0/3156925552 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5a940a2cb0 0x7f5a940a5170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8079a90 tx=0x7f5a9800b410 comp rx=0 tx=0).stop 2024-04-25T01:44:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.093+0000 7f5aad89b700 1 -- 172.21.15.78:0/3156925552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aa8074d80 msgr2=0x7f5aa81a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.093+0000 7f5aad89b700 1 --2- 172.21.15.78:0/3156925552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aa8074d80 0x7f5aa81a0d10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000e9a0 tx=0x7f5a9000ecb0 comp rx=0 tx=0).stop 2024-04-25T01:44:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.093+0000 7f5aad89b700 1 -- 172.21.15.78:0/3156925552 shutdown_connections 2024-04-25T01:44:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.093+0000 7f5aad89b700 1 --2- 172.21.15.78:0/3156925552 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5a940a2cb0 0x7f5a940a5170 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.093+0000 7f5aad89b700 1 --2- 172.21.15.78:0/3156925552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5aa810ec30 0x7f5aa819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.093+0000 7f5aad89b700 1 --2- 172.21.15.78:0/3156925552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5aa8074460 0x7f5aa81a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.093+0000 7f5aad89b700 1 --2- 172.21.15.78:0/3156925552 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5aa8074d80 0x7f5aa81a0d10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.093+0000 7f5aad89b700 1 -- 172.21.15.78:0/3156925552 >> 172.21.15.78:0/3156925552 conn(0x7f5aa8079390 msgr2=0x7f5aa80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.093+0000 7f5aad89b700 1 -- 172.21.15.78:0/3156925552 shutdown_connections 2024-04-25T01:44:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.093+0000 7f5aad89b700 1 -- 172.21.15.78:0/3156925552 wait complete. 2024-04-25T01:44:43.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.948+0000 7fbc31ebf700 1 -- 172.21.15.78:0/480611139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc2c101af0 msgr2=0x7fbc2c10dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:43.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.948+0000 7fbc31ebf700 1 --2- 172.21.15.78:0/480611139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc2c101af0 0x7fbc2c10dfe0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400b0a0 tx=0x7fbc2400b3b0 comp rx=0 tx=0).stop 2024-04-25T01:44:43.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.948+0000 7fbc31ebf700 1 -- 172.21.15.78:0/480611139 shutdown_connections 2024-04-25T01:44:43.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.948+0000 7fbc31ebf700 1 --2- 172.21.15.78:0/480611139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc2c101af0 0x7fbc2c10dfe0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:43.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.948+0000 7fbc31ebf700 1 --2- 172.21.15.78:0/480611139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc2c1071a0 0x7fbc2c107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:43.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.948+0000 7fbc31ebf700 1 --2- 172.21.15.78:0/480611139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c101130 0x7fbc2c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:43.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.948+0000 7fbc31ebf700 1 -- 172.21.15.78:0/480611139 >> 172.21.15.78:0/480611139 conn(0x7fbc2c0780e0 msgr2=0x7fbc2c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:43.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.948+0000 7fbc31ebf700 1 -- 172.21.15.78:0/480611139 shutdown_connections 2024-04-25T01:44:43.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.948+0000 7fbc31ebf700 1 -- 172.21.15.78:0/480611139 wait complete. 2024-04-25T01:44:43.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.949+0000 7fbc31ebf700 1 Processor -- start 2024-04-25T01:44:43.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.949+0000 7fbc31ebf700 1 -- start start 2024-04-25T01:44:43.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.949+0000 7fbc31ebf700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc2c101130 0x7fbc2c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.949+0000 7fbc31ebf700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c101af0 0x7fbc2c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.949+0000 7fbc31ebf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc2c1071a0 0x7fbc2c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.949+0000 7fbc31ebf700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c111510 con 0x7fbc2c101130 2024-04-25T01:44:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.949+0000 7fbc31ebf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c111390 con 0x7fbc2c1071a0 2024-04-25T01:44:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.949+0000 7fbc31ebf700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c111690 con 0x7fbc2c101af0 2024-04-25T01:44:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.949+0000 7fbc2b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc2c101130 0x7fbc2c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.949+0000 7fbc2b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc2c101130 0x7fbc2c19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35584/0 (socket says 172.21.15.78:35584) 2024-04-25T01:44:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.949+0000 7fbc2b7fe700 1 -- 172.21.15.78:0/2053300147 learned_addr learned my addr 172.21.15.78:0/2053300147 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.949+0000 7fbc22ffd700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c101af0 0x7fbc2c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.949+0000 7fbc2bfff700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc2c1071a0 0x7fbc2c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.950+0000 7fbc2b7fe700 1 -- 172.21.15.78:0/2053300147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c101af0 msgr2=0x7fbc2c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:43.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.950+0000 7fbc2b7fe700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c101af0 0x7fbc2c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.950+0000 7fbc2b7fe700 1 -- 172.21.15.78:0/2053300147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc2c1071a0 msgr2=0x7fbc2c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.950+0000 7fbc2b7fe700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc2c1071a0 0x7fbc2c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.950+0000 7fbc2b7fe700 1 -- 172.21.15.78:0/2053300147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c199030 con 0x7fbc2c101130 2024-04-25T01:44:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.950+0000 7fbc22ffd700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c101af0 0x7fbc2c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:44:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.950+0000 7fbc2b7fe700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc2c101130 0x7fbc2c19e6d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00b530 tx=0x7fbc1c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.950+0000 7fbc297fa700 1 -- 172.21.15.78:0/2053300147 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c015020 con 0x7fbc2c101130 2024-04-25T01:44:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.950+0000 7fbc31ebf700 1 -- 172.21.15.78:0/2053300147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2c199320 con 0x7fbc2c101130 2024-04-25T01:44:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.950+0000 7fbc297fa700 1 -- 172.21.15.78:0/2053300147 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc1c011d10 con 0x7fbc2c101130 2024-04-25T01:44:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.950+0000 7fbc297fa700 1 -- 172.21.15.78:0/2053300147 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00f2e0 con 0x7fbc2c101130 2024-04-25T01:44:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.950+0000 7fbc31ebf700 1 -- 172.21.15.78:0/2053300147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2c1a5530 con 0x7fbc2c101130 2024-04-25T01:44:43.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.951+0000 7fbc31ebf700 1 -- 172.21.15.78:0/2053300147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2c102850 con 0x7fbc2c101130 2024-04-25T01:44:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.952+0000 7fbc297fa700 1 -- 172.21.15.78:0/2053300147 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbc1c018020 con 0x7fbc2c101130 2024-04-25T01:44:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.953+0000 7fbc297fa700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbc180a2cb0 0x7fbc180a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.953+0000 7fbc22ffd700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbc180a2cb0 0x7fbc180a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.953+0000 7fbc297fa700 1 -- 172.21.15.78:0/2053300147 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fbc1c0cf0c0 con 0x7fbc2c101130 2024-04-25T01:44:43.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.953+0000 7fbc22ffd700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbc180a2cb0 0x7fbc180a5170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c078650 tx=0x7fbc14008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:43.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:43.957+0000 7fbc297fa700 1 -- 172.21.15.78:0/2053300147 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbc1c01f020 con 0x7fbc2c101130 2024-04-25T01:44:44.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.142+0000 7f529ddb4700 1 -- 172.21.15.78:0/1431331565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5298100820 msgr2=0x7f5298100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.142+0000 7f529ddb4700 1 --2- 172.21.15.78:0/1431331565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5298100820 0x7f5298100c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f528c009990 tx=0x7f528c009ca0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.143+0000 7f529ddb4700 1 -- 172.21.15.78:0/1431331565 shutdown_connections 2024-04-25T01:44:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.143+0000 7f529ddb4700 1 --2- 172.21.15.78:0/1431331565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52981082c0 0x7f52981086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.143+0000 7f529ddb4700 1 --2- 172.21.15.78:0/1431331565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52981011c0 0x7f529810f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.143+0000 7f529ddb4700 1 --2- 172.21.15.78:0/1431331565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5298100820 0x7f5298100c80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.143+0000 7f529ddb4700 1 -- 172.21.15.78:0/1431331565 >> 172.21.15.78:0/1431331565 conn(0x7f52980777a0 msgr2=0x7f52980feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.143+0000 7f529ddb4700 1 -- 172.21.15.78:0/1431331565 shutdown_connections 2024-04-25T01:44:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.143+0000 7f529ddb4700 1 -- 172.21.15.78:0/1431331565 wait complete. 2024-04-25T01:44:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.143+0000 7f529ddb4700 1 Processor -- start 2024-04-25T01:44:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.143+0000 7f529ddb4700 1 -- start start 2024-04-25T01:44:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f529ddb4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5298100820 0x7f529819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f529ddb4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52981011c0 0x7f529819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f529ddb4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52981082c0 0x7f52981987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f529ddb4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5298112520 con 0x7f52981082c0 2024-04-25T01:44:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f529ddb4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f52981123a0 con 0x7f5298100820 2024-04-25T01:44:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f529ddb4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52981126a0 con 0x7f52981011c0 2024-04-25T01:44:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f5297fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52981082c0 0x7f52981987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f5297fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52981082c0 0x7f52981987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35604/0 (socket says 172.21.15.78:35604) 2024-04-25T01:44:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f5297fff700 1 -- 172.21.15.78:0/3257468468 learned_addr learned my addr 172.21.15.78:0/3257468468 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f52977fe700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5298100820 0x7f529819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f5296ffd700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52981011c0 0x7f529819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f5297fff700 1 -- 172.21.15.78:0/3257468468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52981011c0 msgr2=0x7f529819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f5297fff700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52981011c0 0x7f529819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f5297fff700 1 -- 172.21.15.78:0/3257468468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5298100820 msgr2=0x7f529819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f5297fff700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5298100820 0x7f529819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f5297fff700 1 -- 172.21.15.78:0/3257468468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5298198f80 con 0x7f52981082c0 2024-04-25T01:44:44.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f52977fe700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5298100820 0x7f529819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:44:44.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.144+0000 7f5297fff700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52981082c0 0x7f52981987a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5290007ee0 tx=0x7f529000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:44.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.145+0000 7f5294ff9700 1 -- 172.21.15.78:0/3257468468 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529000dbd0 con 0x7f52981082c0 2024-04-25T01:44:44.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.145+0000 7f529ddb4700 1 -- 172.21.15.78:0/3257468468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5298199270 con 0x7f52981082c0 2024-04-25T01:44:44.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.145+0000 7f5294ff9700 1 -- 172.21.15.78:0/3257468468 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f529000dd70 con 0x7f52981082c0 2024-04-25T01:44:44.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.145+0000 7f5294ff9700 1 -- 172.21.15.78:0/3257468468 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52900133f0 con 0x7f52981082c0 2024-04-25T01:44:44.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.145+0000 7f529ddb4700 1 -- 172.21.15.78:0/3257468468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5298101930 con 0x7f52981082c0 2024-04-25T01:44:44.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.146+0000 7f529ddb4700 1 -- 172.21.15.78:0/3257468468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5298068f20 con 0x7f52981082c0 2024-04-25T01:44:44.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.147+0000 7f5294ff9700 1 -- 172.21.15.78:0/3257468468 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f529001a020 con 0x7f52981082c0 2024-04-25T01:44:44.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.147+0000 7f5294ff9700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f52800a2cb0 0x7f52800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:44.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.148+0000 7f52977fe700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f52800a2cb0 0x7f52800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:44.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.148+0000 7f5294ff9700 1 -- 172.21.15.78:0/3257468468 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f52900cf3e0 con 0x7f52981082c0 2024-04-25T01:44:44.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.148+0000 7f52977fe700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f52800a2cb0 0x7f52800a5170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5288005e10 tx=0x7f5288005d60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:44.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.151+0000 7f5294ff9700 1 -- 172.21.15.78:0/3257468468 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5290010070 con 0x7f52981082c0 2024-04-25T01:44:44.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.168+0000 7fbc31ebf700 1 -- 172.21.15.78:0/2053300147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbc2c1a57f0 con 0x7fbc2c101130 2024-04-25T01:44:44.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.169+0000 7fbc297fa700 1 -- 172.21.15.78:0/2053300147 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbc1c09bc90 con 0x7fbc2c101130 2024-04-25T01:44:44.161 INFO:teuthology.orchestra.run.smithi078.stdout:201863462968 2024-04-25T01:44:44.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.171+0000 7fbc31ebf700 1 -- 172.21.15.78:0/2053300147 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbc180a2cb0 msgr2=0x7fbc180a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:44.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.171+0000 7fbc31ebf700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbc180a2cb0 0x7fbc180a5170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c078650 tx=0x7fbc14008040 comp rx=0 tx=0).stop 2024-04-25T01:44:44.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.171+0000 7fbc31ebf700 1 -- 172.21.15.78:0/2053300147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc2c101130 msgr2=0x7fbc2c19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:44.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.171+0000 7fbc31ebf700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc2c101130 0x7fbc2c19e6d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00b530 tx=0x7fbc1c00b840 comp rx=0 tx=0).stop 2024-04-25T01:44:44.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.171+0000 7fbc31ebf700 1 -- 172.21.15.78:0/2053300147 shutdown_connections 2024-04-25T01:44:44.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.171+0000 7fbc31ebf700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fbc180a2cb0 0x7fbc180a5170 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.171+0000 7fbc31ebf700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc2c1071a0 0x7fbc2c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.171+0000 7fbc31ebf700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc2c101130 0x7fbc2c19e6d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.171+0000 7fbc31ebf700 1 --2- 172.21.15.78:0/2053300147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc2c101af0 0x7fbc2c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.171+0000 7fbc31ebf700 1 -- 172.21.15.78:0/2053300147 >> 172.21.15.78:0/2053300147 conn(0x7fbc2c0780e0 msgr2=0x7fbc2c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:44.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.171+0000 7fbc31ebf700 1 -- 172.21.15.78:0/2053300147 shutdown_connections 2024-04-25T01:44:44.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.171+0000 7fbc31ebf700 1 -- 172.21.15.78:0/2053300147 wait complete. 2024-04-25T01:44:44.176 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:44 smithi078 ceph-mon[29469]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:44:44.176 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:44 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3156925552' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T01:44:44.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:44 smithi152 ceph-mon[32465]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:44:44.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:44 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3156925552' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T01:44:44.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.368+0000 7f529ddb4700 1 -- 172.21.15.78:0/3257468468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f529802dd90 con 0x7f52981082c0 2024-04-25T01:44:44.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.368+0000 7f5294ff9700 1 -- 172.21.15.78:0/3257468468 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f529009bea0 con 0x7f52981082c0 2024-04-25T01:44:44.361 INFO:teuthology.orchestra.run.smithi078.stdout:257698037806 2024-04-25T01:44:44.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.370+0000 7f529ddb4700 1 -- 172.21.15.78:0/3257468468 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f52800a2cb0 msgr2=0x7f52800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:44.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.370+0000 7f529ddb4700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f52800a2cb0 0x7f52800a5170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5288005e10 tx=0x7f5288005d60 comp rx=0 tx=0).stop 2024-04-25T01:44:44.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.370+0000 7f529ddb4700 1 -- 172.21.15.78:0/3257468468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52981082c0 msgr2=0x7f52981987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:44.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.370+0000 7f529ddb4700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52981082c0 0x7f52981987a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5290007ee0 tx=0x7f529000d3b0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.371+0000 7f529ddb4700 1 -- 172.21.15.78:0/3257468468 shutdown_connections 2024-04-25T01:44:44.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.371+0000 7f529ddb4700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f52800a2cb0 0x7f52800a5170 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.371+0000 7f529ddb4700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52981082c0 0x7f52981987a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.371+0000 7f529ddb4700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5298100820 0x7f529819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.371+0000 7f529ddb4700 1 --2- 172.21.15.78:0/3257468468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52981011c0 0x7f529819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.371+0000 7f529ddb4700 1 -- 172.21.15.78:0/3257468468 >> 172.21.15.78:0/3257468468 conn(0x7f52980777a0 msgr2=0x7f52980feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:44.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.371+0000 7f529ddb4700 1 -- 172.21.15.78:0/3257468468 shutdown_connections 2024-04-25T01:44:44.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.371+0000 7f529ddb4700 1 -- 172.21.15.78:0/3257468468 wait complete. 2024-04-25T01:44:44.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:44 smithi196 ceph-mon[32978]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:44:44.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:44 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3156925552' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T01:44:44.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.421+0000 7f179533d700 1 -- 172.21.15.78:0/323686765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901011c0 msgr2=0x7f179010f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:44.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.421+0000 7f179533d700 1 --2- 172.21.15.78:0/323686765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901011c0 0x7f179010f100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f178800b0a0 tx=0x7f178800b3b0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.421+0000 7f179533d700 1 -- 172.21.15.78:0/323686765 shutdown_connections 2024-04-25T01:44:44.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.421+0000 7f179533d700 1 --2- 172.21.15.78:0/323686765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17901082c0 0x7f17901086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.421+0000 7f179533d700 1 --2- 172.21.15.78:0/323686765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901011c0 0x7f179010f100 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.421+0000 7f179533d700 1 --2- 172.21.15.78:0/323686765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1790100820 0x7f1790100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.421+0000 7f179533d700 1 -- 172.21.15.78:0/323686765 >> 172.21.15.78:0/323686765 conn(0x7f17900777a0 msgr2=0x7f17900feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:44.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.421+0000 7f179533d700 1 -- 172.21.15.78:0/323686765 shutdown_connections 2024-04-25T01:44:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.421+0000 7f179533d700 1 -- 172.21.15.78:0/323686765 wait complete. 2024-04-25T01:44:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f179533d700 1 Processor -- start 2024-04-25T01:44:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f179533d700 1 -- start start 2024-04-25T01:44:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f179533d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1790100820 0x7f1790101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f179533d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17901011c0 0x7f1790101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f179533d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901082c0 0x7f1790102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f179533d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1790112540 con 0x7f17901082c0 2024-04-25T01:44:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f179533d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f17901123c0 con 0x7f1790100820 2024-04-25T01:44:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f179533d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17901126c0 con 0x7f17901011c0 2024-04-25T01:44:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f178f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901082c0 0x7f1790102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f178f7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901082c0 0x7f1790102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35624/0 (socket says 172.21.15.78:35624) 2024-04-25T01:44:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f178f7fe700 1 -- 172.21.15.78:0/3869850971 learned_addr learned my addr 172.21.15.78:0/3869850971 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f178e7fc700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17901011c0 0x7f1790101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f178effd700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1790100820 0x7f1790101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f178f7fe700 1 -- 172.21.15.78:0/3869850971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17901011c0 msgr2=0x7f1790101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f178f7fe700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17901011c0 0x7f1790101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f178f7fe700 1 -- 172.21.15.78:0/3869850971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1790100820 msgr2=0x7f1790101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f178f7fe700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1790100820 0x7f1790101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f178f7fe700 1 -- 172.21.15.78:0/3869850971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17901a6e80 con 0x7f17901082c0 2024-04-25T01:44:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.422+0000 7f178effd700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1790100820 0x7f1790101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:44:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.423+0000 7f178f7fe700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901082c0 0x7f1790102480 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f178800b780 tx=0x7f1788004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.423+0000 7f1777fff700 1 -- 172.21.15.78:0/3869850971 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788021070 con 0x7f17901082c0 2024-04-25T01:44:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.423+0000 7f179533d700 1 -- 172.21.15.78:0/3869850971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17901a7110 con 0x7f17901082c0 2024-04-25T01:44:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.423+0000 7f1777fff700 1 -- 172.21.15.78:0/3869850971 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1788007b80 con 0x7f17901082c0 2024-04-25T01:44:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.423+0000 7f1777fff700 1 -- 172.21.15.78:0/3869850971 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788005370 con 0x7f17901082c0 2024-04-25T01:44:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.423+0000 7f179533d700 1 -- 172.21.15.78:0/3869850971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17901a7610 con 0x7f17901082c0 2024-04-25T01:44:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.424+0000 7f179533d700 1 -- 172.21.15.78:0/3869850971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1790103b40 con 0x7f17901082c0 2024-04-25T01:44:44.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.425+0000 7f1777fff700 1 -- 172.21.15.78:0/3869850971 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1788024070 con 0x7f17901082c0 2024-04-25T01:44:44.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.426+0000 7f1777fff700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17780a2cb0 0x7f17780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:44.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.426+0000 7f178effd700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17780a2cb0 0x7f17780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:44.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.426+0000 7f1777fff700 1 -- 172.21.15.78:0/3869850971 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f178800f020 con 0x7f17901082c0 2024-04-25T01:44:44.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.426+0000 7f178effd700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17780a2cb0 0x7f17780a5170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1780009bc0 tx=0x7f1780009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:44.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.429+0000 7f1777fff700 1 -- 172.21.15.78:0/3869850971 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f178802b020 con 0x7f17901082c0 2024-04-25T01:44:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.637+0000 7f179533d700 1 -- 172.21.15.78:0/3869850971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f179002dcf0 con 0x7f17901082c0 2024-04-25T01:44:44.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.637+0000 7f1777fff700 1 -- 172.21.15.78:0/3869850971 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f178809d980 con 0x7f17901082c0 2024-04-25T01:44:44.630 INFO:teuthology.orchestra.run.smithi078.stdout:167503724605 2024-04-25T01:44:44.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.640+0000 7f179533d700 1 -- 172.21.15.78:0/3869850971 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17780a2cb0 msgr2=0x7f17780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:44.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.640+0000 7f179533d700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17780a2cb0 0x7f17780a5170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1780009bc0 tx=0x7f1780009260 comp rx=0 tx=0).stop 2024-04-25T01:44:44.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.640+0000 7f179533d700 1 -- 172.21.15.78:0/3869850971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901082c0 msgr2=0x7f1790102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:44.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.640+0000 7f179533d700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901082c0 0x7f1790102480 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f178800b780 tx=0x7f1788004d00 comp rx=0 tx=0).stop 2024-04-25T01:44:44.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.640+0000 7f179533d700 1 -- 172.21.15.78:0/3869850971 shutdown_connections 2024-04-25T01:44:44.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.640+0000 7f179533d700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f17780a2cb0 0x7f17780a5170 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.640+0000 7f179533d700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17901082c0 0x7f1790102480 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.640+0000 7f179533d700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1790100820 0x7f1790101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.640+0000 7f179533d700 1 --2- 172.21.15.78:0/3869850971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17901011c0 0x7f1790101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:44.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.640+0000 7f179533d700 1 -- 172.21.15.78:0/3869850971 >> 172.21.15.78:0/3869850971 conn(0x7f17900777a0 msgr2=0x7f17900fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:44.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.640+0000 7f179533d700 1 -- 172.21.15.78:0/3869850971 shutdown_connections 2024-04-25T01:44:44.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:44.640+0000 7f179533d700 1 -- 172.21.15.78:0/3869850971 wait complete. 2024-04-25T01:44:45.079 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182466 got 107374182470 for osd.3 2024-04-25T01:44:45.079 DEBUG:teuthology.parallel:result is None 2024-04-25T01:44:45.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:45 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2053300147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T01:44:45.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:45 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3257468468' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T01:44:45.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:45 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3869850971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T01:44:45.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:45 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2053300147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T01:44:45.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:45 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3257468468' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T01:44:45.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:45 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3869850971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T01:44:45.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:45 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2053300147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T01:44:45.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:45 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3257468468' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T01:44:45.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:45 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3869850971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T01:44:45.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.910+0000 7f1aca556700 1 -- 172.21.15.78:0/3965734322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac4106470 msgr2=0x7f1ac410e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:45.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.910+0000 7f1aca556700 1 --2- 172.21.15.78:0/3965734322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac4106470 0x7f1ac410e040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00b0d0 tx=0x7f1abc00b3e0 comp rx=0 tx=0).stop 2024-04-25T01:44:45.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.910+0000 7f1aca556700 1 -- 172.21.15.78:0/3965734322 shutdown_connections 2024-04-25T01:44:45.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.910+0000 7f1aca556700 1 --2- 172.21.15.78:0/3965734322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac4106470 0x7f1ac410e040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:45.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.910+0000 7f1aca556700 1 --2- 172.21.15.78:0/3965734322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4101420 0x7f1ac4101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:45.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.910+0000 7f1aca556700 1 --2- 172.21.15.78:0/3965734322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac4101dd0 0x7f1ac4105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:45.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.910+0000 7f1aca556700 1 -- 172.21.15.78:0/3965734322 >> 172.21.15.78:0/3965734322 conn(0x7f1ac4077180 msgr2=0x7f1ac4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:45.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.910+0000 7f1aca556700 1 -- 172.21.15.78:0/3965734322 shutdown_connections 2024-04-25T01:44:45.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.910+0000 7f1aca556700 1 -- 172.21.15.78:0/3965734322 wait complete. 2024-04-25T01:44:45.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.911+0000 7f1aca556700 1 Processor -- start 2024-04-25T01:44:45.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.911+0000 7f1aca556700 1 -- start start 2024-04-25T01:44:45.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.911+0000 7f1aca556700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac4101420 0x7f1ac4102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:45.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.911+0000 7f1aca556700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4101dd0 0x7f1ac4103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:45.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.911+0000 7f1aca556700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac4106470 0x7f1ac41038b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:45.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.911+0000 7f1aca556700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4111480 con 0x7f1ac4101420 2024-04-25T01:44:45.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.911+0000 7f1aca556700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4111300 con 0x7f1ac4101dd0 2024-04-25T01:44:45.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.911+0000 7f1aca556700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4111600 con 0x7f1ac4106470 2024-04-25T01:44:45.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.911+0000 7f1ac3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac4101420 0x7f1ac4102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:45.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac3fff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac4101420 0x7f1ac4102e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35640/0 (socket says 172.21.15.78:35640) 2024-04-25T01:44:45.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac3fff700 1 -- 172.21.15.78:0/3872027060 learned_addr learned my addr 172.21.15.78:0/3872027060 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:45.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac37fe700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4101dd0 0x7f1ac4103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:45.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac8af3700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac4106470 0x7f1ac41038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:45.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac3fff700 1 -- 172.21.15.78:0/3872027060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac4106470 msgr2=0x7f1ac41038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:45.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac3fff700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac4106470 0x7f1ac41038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:45.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac3fff700 1 -- 172.21.15.78:0/3872027060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4101dd0 msgr2=0x7f1ac4103370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:45.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac3fff700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4101dd0 0x7f1ac4103370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:45.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac3fff700 1 -- 172.21.15.78:0/3872027060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac41a6eb0 con 0x7f1ac4101420 2024-04-25T01:44:45.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac8af3700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac4106470 0x7f1ac41038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:44:45.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac37fe700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4101dd0 0x7f1ac4103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:44:45.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac3fff700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac4101420 0x7f1ac4102e30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400d950 tx=0x7f1ab400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:45.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac17fa700 1 -- 172.21.15.78:0/3872027060 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4016070 con 0x7f1ac4101420 2024-04-25T01:44:45.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1aca556700 1 -- 172.21.15.78:0/3872027060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac41a71a0 con 0x7f1ac4101420 2024-04-25T01:44:45.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1aca556700 1 -- 172.21.15.78:0/3872027060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac41a7730 con 0x7f1ac4101420 2024-04-25T01:44:45.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac17fa700 1 -- 172.21.15.78:0/3872027060 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab4009b40 con 0x7f1ac4101420 2024-04-25T01:44:45.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.912+0000 7f1ac17fa700 1 -- 172.21.15.78:0/3872027060 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400f620 con 0x7f1ac4101420 2024-04-25T01:44:45.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.913+0000 7f1aca556700 1 -- 172.21.15.78:0/3872027060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac4068f20 con 0x7f1ac4101420 2024-04-25T01:44:45.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.914+0000 7f1ac17fa700 1 -- 172.21.15.78:0/3872027060 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1ab4019070 con 0x7f1ac4101420 2024-04-25T01:44:45.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.915+0000 7f1ac17fa700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1aac0a2d30 0x7f1aac0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:45.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.915+0000 7f1ac37fe700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1aac0a2d30 0x7f1aac0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:45.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.915+0000 7f1ac17fa700 1 -- 172.21.15.78:0/3872027060 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f1ab40cf580 con 0x7f1ac4101420 2024-04-25T01:44:45.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.915+0000 7f1ac37fe700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1aac0a2d30 0x7f1aac0a51f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005e10 tx=0x7f1ab8005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:45.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:45.919+0000 7f1ac17fa700 1 -- 172.21.15.78:0/3872027060 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1ab409c040 con 0x7f1ac4101420 2024-04-25T01:44:46.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:46.120+0000 7f1aca556700 1 -- 172.21.15.78:0/3872027060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1ac4104360 con 0x7f1ac4101420 2024-04-25T01:44:46.113 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:46 smithi078 ceph-mon[29469]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:44:46.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:46.120+0000 7f1ac17fa700 1 -- 172.21.15.78:0/3872027060 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1ab409b940 con 0x7f1ac4101420 2024-04-25T01:44:46.113 INFO:teuthology.orchestra.run.smithi078.stdout:141733920835 2024-04-25T01:44:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:46.123+0000 7f1aca556700 1 -- 172.21.15.78:0/3872027060 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1aac0a2d30 msgr2=0x7f1aac0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:46.123+0000 7f1aca556700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1aac0a2d30 0x7f1aac0a51f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005e10 tx=0x7f1ab8005d80 comp rx=0 tx=0).stop 2024-04-25T01:44:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:46.123+0000 7f1aca556700 1 -- 172.21.15.78:0/3872027060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac4101420 msgr2=0x7f1ac4102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:46.123+0000 7f1aca556700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac4101420 0x7f1ac4102e30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400d950 tx=0x7f1ab400dd10 comp rx=0 tx=0).stop 2024-04-25T01:44:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:46.123+0000 7f1aca556700 1 -- 172.21.15.78:0/3872027060 shutdown_connections 2024-04-25T01:44:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:46.123+0000 7f1aca556700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1aac0a2d30 0x7f1aac0a51f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:46.123+0000 7f1aca556700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ac4106470 0x7f1ac41038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:46.123+0000 7f1aca556700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac4101420 0x7f1ac4102e30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:46.123+0000 7f1aca556700 1 --2- 172.21.15.78:0/3872027060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4101dd0 0x7f1ac4103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:46.123+0000 7f1aca556700 1 -- 172.21.15.78:0/3872027060 >> 172.21.15.78:0/3872027060 conn(0x7f1ac4077180 msgr2=0x7f1ac406ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:46.123+0000 7f1aca556700 1 -- 172.21.15.78:0/3872027060 shutdown_connections 2024-04-25T01:44:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:46.123+0000 7f1aca556700 1 -- 172.21.15.78:0/3872027060 wait complete. 2024-04-25T01:44:46.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:46 smithi152 ceph-mon[32465]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:44:46.366 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:46 smithi196 ceph-mon[32978]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:44:46.516 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724597 got 167503724605 for osd.5 2024-04-25T01:44:46.516 DEBUG:teuthology.parallel:result is None 2024-04-25T01:44:47.145 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574924 got 55834574928 for osd.1 2024-04-25T01:44:47.145 DEBUG:teuthology.parallel:result is None 2024-04-25T01:44:47.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:47 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3872027060' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T01:44:47.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:47 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3872027060' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T01:44:47.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:47 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3872027060' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T01:44:47.581 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037800 got 257698037806 for osd.8 2024-04-25T01:44:47.582 DEBUG:teuthology.parallel:result is None 2024-04-25T01:44:48.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:48 smithi078 ceph-mon[29469]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:44:48.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:48 smithi078 ceph-mon[29469]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T01:44:48.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:48 smithi152 ceph-mon[32465]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:44:48.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:48 smithi152 ceph-mon[32465]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T01:44:48.314 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462963 got 201863462968 for osd.6 2024-04-25T01:44:48.314 DEBUG:teuthology.parallel:result is None 2024-04-25T01:44:48.349 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:48.349 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:48.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:48 smithi196 ceph-mon[32978]: pgmap v337: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:44:48.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:48 smithi196 ceph-mon[32978]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T01:44:48.846 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920828 got 141733920835 for osd.4 2024-04-25T01:44:48.846 DEBUG:teuthology.parallel:result is None 2024-04-25T01:44:49.757 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:49.757 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:49.758 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:49 smithi078 ceph-mon[29469]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:44:49.758 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:44:49.758 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:44:49.758 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:49.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:49.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:49.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:49 smithi152 ceph-mon[32465]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:44:49.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:44:49.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:44:49.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:49.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:49.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:49.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:49 smithi196 ceph-mon[32978]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:44:49.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:44:49.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:44:49.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:44:51.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.644+0000 7f2070c2d700 1 -- 172.21.15.78:0/1422712394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f206c100bd0 msgr2=0x7f206c101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.644+0000 7f2070c2d700 1 --2- 172.21.15.78:0/1422712394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f206c100bd0 0x7f206c101050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2060009990 tx=0x7f2060009ca0 comp rx=0 tx=0).stop 2024-04-25T01:44:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.644+0000 7f2070c2d700 1 -- 172.21.15.78:0/1422712394 shutdown_connections 2024-04-25T01:44:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.644+0000 7f2070c2d700 1 --2- 172.21.15.78:0/1422712394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f206c106980 0x7f206c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.644+0000 7f2070c2d700 1 --2- 172.21.15.78:0/1422712394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f206c101590 0x7f206c10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.644+0000 7f2070c2d700 1 --2- 172.21.15.78:0/1422712394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f206c100bd0 0x7f206c101050 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.644+0000 7f2070c2d700 1 -- 172.21.15.78:0/1422712394 >> 172.21.15.78:0/1422712394 conn(0x7f206c076450 msgr2=0x7f206c076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.644+0000 7f2070c2d700 1 -- 172.21.15.78:0/1422712394 shutdown_connections 2024-04-25T01:44:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.644+0000 7f2070c2d700 1 -- 172.21.15.78:0/1422712394 wait complete. 2024-04-25T01:44:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f2070c2d700 1 Processor -- start 2024-04-25T01:44:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f2070c2d700 1 -- start start 2024-04-25T01:44:51.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f2070c2d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f206c100bd0 0x7f206c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f2070c2d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f206c101590 0x7f206c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f2070c2d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f206c106980 0x7f206c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f2070c2d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f206c110360 con 0x7f206c106980 2024-04-25T01:44:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f2070c2d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f206c1101e0 con 0x7f206c100bd0 2024-04-25T01:44:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f2070c2d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f206c1104e0 con 0x7f206c101590 2024-04-25T01:44:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f206ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f206c106980 0x7f206c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f206ad9d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f206c106980 0x7f206c1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37386/0 (socket says 172.21.15.78:37386) 2024-04-25T01:44:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f206ad9d700 1 -- 172.21.15.78:0/3371548904 learned_addr learned my addr 172.21.15.78:0/3371548904 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f2069d9b700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f206c101590 0x7f206c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f206a59c700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f206c100bd0 0x7f206c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f206ad9d700 1 -- 172.21.15.78:0/3371548904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f206c101590 msgr2=0x7f206c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f206ad9d700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f206c101590 0x7f206c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:51.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f206ad9d700 1 -- 172.21.15.78:0/3371548904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f206c100bd0 msgr2=0x7f206c19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f206ad9d700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f206c100bd0 0x7f206c19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f206ad9d700 1 -- 172.21.15.78:0/3371548904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206c198f90 con 0x7f206c106980 2024-04-25T01:44:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f2069d9b700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f206c101590 0x7f206c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:44:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.645+0000 7f206ad9d700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f206c106980 0x7f206c1987b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f206400d3a0 tx=0x7f206400d760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.646+0000 7f205b7fe700 1 -- 172.21.15.78:0/3371548904 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064017020 con 0x7f206c106980 2024-04-25T01:44:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.646+0000 7f205b7fe700 1 -- 172.21.15.78:0/3371548904 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206400de30 con 0x7f206c106980 2024-04-25T01:44:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.646+0000 7f2070c2d700 1 -- 172.21.15.78:0/3371548904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f206c199280 con 0x7f206c106980 2024-04-25T01:44:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.646+0000 7f205b7fe700 1 -- 172.21.15.78:0/3371548904 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064008b70 con 0x7f206c106980 2024-04-25T01:44:51.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.646+0000 7f2070c2d700 1 -- 172.21.15.78:0/3371548904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f206c1a5490 con 0x7f206c106980 2024-04-25T01:44:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.647+0000 7f2070c2d700 1 -- 172.21.15.78:0/3371548904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f206c102010 con 0x7f206c106980 2024-04-25T01:44:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.648+0000 7f205b7fe700 1 -- 172.21.15.78:0/3371548904 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2064012070 con 0x7f206c106980 2024-04-25T01:44:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.648+0000 7f205b7fe700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f20540a2ca0 0x7f20540a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.648+0000 7f206a59c700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f20540a2ca0 0x7f20540a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.648+0000 7f205b7fe700 1 -- 172.21.15.78:0/3371548904 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f20640cef70 con 0x7f206c106980 2024-04-25T01:44:51.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.649+0000 7f206a59c700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f20540a2ca0 0x7f20540a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f205c005fd0 tx=0x7f205c00af20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:51.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.653+0000 7f205b7fe700 1 -- 172.21.15.78:0/3371548904 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2064021020 con 0x7f206c106980 2024-04-25T01:44:51.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.852+0000 7f2070c2d700 1 -- 172.21.15.78:0/3371548904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f206c02db30 con 0x7f206c106980 2024-04-25T01:44:51.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.853+0000 7f205b7fe700 1 -- 172.21.15.78:0/3371548904 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f206409bc20 con 0x7f206c106980 2024-04-25T01:44:51.846 INFO:teuthology.orchestra.run.smithi078.stdout:223338299444 2024-04-25T01:44:51.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.855+0000 7f2070c2d700 1 -- 172.21.15.78:0/3371548904 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f20540a2ca0 msgr2=0x7f20540a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.855+0000 7f2070c2d700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f20540a2ca0 0x7f20540a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f205c005fd0 tx=0x7f205c00af20 comp rx=0 tx=0).stop 2024-04-25T01:44:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.855+0000 7f2070c2d700 1 -- 172.21.15.78:0/3371548904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f206c106980 msgr2=0x7f206c1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.855+0000 7f2070c2d700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f206c106980 0x7f206c1987b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f206400d3a0 tx=0x7f206400d760 comp rx=0 tx=0).stop 2024-04-25T01:44:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.855+0000 7f2070c2d700 1 -- 172.21.15.78:0/3371548904 shutdown_connections 2024-04-25T01:44:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.855+0000 7f2070c2d700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f20540a2ca0 0x7f20540a5160 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.855+0000 7f2070c2d700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f206c106980 0x7f206c1987b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.855+0000 7f2070c2d700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f206c100bd0 0x7f206c19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.855+0000 7f2070c2d700 1 --2- 172.21.15.78:0/3371548904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f206c101590 0x7f206c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.855+0000 7f2070c2d700 1 -- 172.21.15.78:0/3371548904 >> 172.21.15.78:0/3371548904 conn(0x7f206c076450 msgr2=0x7f206c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.855+0000 7f2070c2d700 1 -- 172.21.15.78:0/3371548904 shutdown_connections 2024-04-25T01:44:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:51.855+0000 7f2070c2d700 1 -- 172.21.15.78:0/3371548904 wait complete. 2024-04-25T01:44:52.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:51 smithi078 ceph-mon[29469]: pgmap v339: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-25T01:44:52.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:51 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3371548904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T01:44:52.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:51 smithi152 ceph-mon[32465]: pgmap v339: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-25T01:44:52.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:51 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3371548904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T01:44:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.331+0000 7fb441528700 1 -- 172.21.15.78:0/1317453807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb43c1012c0 msgr2=0x7fb43c110fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.331+0000 7fb441528700 1 --2- 172.21.15.78:0/1317453807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb43c1012c0 0x7fb43c110fb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb43400b0a0 tx=0x7fb43400b3b0 comp rx=0 tx=0).stop 2024-04-25T01:44:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.331+0000 7fb441528700 1 -- 172.21.15.78:0/1317453807 shutdown_connections 2024-04-25T01:44:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.331+0000 7fb441528700 1 --2- 172.21.15.78:0/1317453807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43c103140 0x7fb43c103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.331+0000 7fb441528700 1 --2- 172.21.15.78:0/1317453807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb43c1012c0 0x7fb43c110fb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.331+0000 7fb441528700 1 --2- 172.21.15.78:0/1317453807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43c100820 0x7fb43c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.331+0000 7fb441528700 1 -- 172.21.15.78:0/1317453807 >> 172.21.15.78:0/1317453807 conn(0x7fb43c0777a0 msgr2=0x7fb43c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.331+0000 7fb441528700 1 -- 172.21.15.78:0/1317453807 shutdown_connections 2024-04-25T01:44:52.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.331+0000 7fb441528700 1 -- 172.21.15.78:0/1317453807 wait complete. 2024-04-25T01:44:52.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.332+0000 7fb441528700 1 Processor -- start 2024-04-25T01:44:52.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.332+0000 7fb441528700 1 -- start start 2024-04-25T01:44:52.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.332+0000 7fb441528700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43c100820 0x7fb43c1a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:52.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.332+0000 7fb441528700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb43c1012c0 0x7fb43c1a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:52.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.332+0000 7fb441528700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43c103140 0x7fb43c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:52.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.332+0000 7fb441528700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1147a0 con 0x7fb43c1012c0 2024-04-25T01:44:52.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.332+0000 7fb441528700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c114620 con 0x7fb43c100820 2024-04-25T01:44:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.332+0000 7fb441528700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c114920 con 0x7fb43c103140 2024-04-25T01:44:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.332+0000 7fb4327fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb43c1012c0 0x7fb43c1a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.332+0000 7fb4327fc700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb43c1012c0 0x7fb43c1a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37400/0 (socket says 172.21.15.78:37400) 2024-04-25T01:44:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.332+0000 7fb4327fc700 1 -- 172.21.15.78:0/3963960176 learned_addr learned my addr 172.21.15.78:0/3963960176 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.333+0000 7fb43b7fe700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43c103140 0x7fb43c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.333+0000 7fb43affd700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43c100820 0x7fb43c1a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.333+0000 7fb4327fc700 1 -- 172.21.15.78:0/3963960176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43c103140 msgr2=0x7fb43c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.333+0000 7fb4327fc700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43c103140 0x7fb43c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.333+0000 7fb4327fc700 1 -- 172.21.15.78:0/3963960176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43c100820 msgr2=0x7fb43c1a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.333+0000 7fb4327fc700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43c100820 0x7fb43c1a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.333+0000 7fb4327fc700 1 -- 172.21.15.78:0/3963960176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43c19b160 con 0x7fb43c1012c0 2024-04-25T01:44:52.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.333+0000 7fb43affd700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43c100820 0x7fb43c1a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:44:52.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.333+0000 7fb4327fc700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb43c1012c0 0x7fb43c1a0d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb42400b530 tx=0x7fb42400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:52.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.333+0000 7fb438ff9700 1 -- 172.21.15.78:0/3963960176 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424015020 con 0x7fb43c1012c0 2024-04-25T01:44:52.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.333+0000 7fb438ff9700 1 -- 172.21.15.78:0/3963960176 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb424011d10 con 0x7fb43c1012c0 2024-04-25T01:44:52.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.333+0000 7fb441528700 1 -- 172.21.15.78:0/3963960176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43c19b450 con 0x7fb43c1012c0 2024-04-25T01:44:52.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.333+0000 7fb438ff9700 1 -- 172.21.15.78:0/3963960176 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424010e50 con 0x7fb43c1012c0 2024-04-25T01:44:52.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.333+0000 7fb441528700 1 -- 172.21.15.78:0/3963960176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb43c1a7710 con 0x7fb43c1012c0 2024-04-25T01:44:52.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.334+0000 7fb441528700 1 -- 172.21.15.78:0/3963960176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb43c101830 con 0x7fb43c1012c0 2024-04-25T01:44:52.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.335+0000 7fb438ff9700 1 -- 172.21.15.78:0/3963960176 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb424018020 con 0x7fb43c1012c0 2024-04-25T01:44:52.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.336+0000 7fb438ff9700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb4280a2dc0 0x7fb4280a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:52.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.336+0000 7fb43affd700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb4280a2dc0 0x7fb4280a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:52.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.336+0000 7fb438ff9700 1 -- 172.21.15.78:0/3963960176 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fb4240cf120 con 0x7fb43c1012c0 2024-04-25T01:44:52.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.336+0000 7fb43affd700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb4280a2dc0 0x7fb4280a5280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb42c005fd0 tx=0x7fb42c005de0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:52.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.340+0000 7fb438ff9700 1 -- 172.21.15.78:0/3963960176 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb42401f020 con 0x7fb43c1012c0 2024-04-25T01:44:52.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:51 smithi196 ceph-mon[32978]: pgmap v339: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-25T01:44:52.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:51 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3371548904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T01:44:52.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.547+0000 7fb441528700 1 -- 172.21.15.78:0/3963960176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb43c19bf30 con 0x7fb43c1012c0 2024-04-25T01:44:52.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.548+0000 7fb438ff9700 1 -- 172.21.15.78:0/3963960176 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb424097070 con 0x7fb43c1012c0 2024-04-25T01:44:52.541 INFO:teuthology.orchestra.run.smithi078.stdout:77309411406 2024-04-25T01:44:52.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.550+0000 7fb441528700 1 -- 172.21.15.78:0/3963960176 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb4280a2dc0 msgr2=0x7fb4280a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:52.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.550+0000 7fb441528700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb4280a2dc0 0x7fb4280a5280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb42c005fd0 tx=0x7fb42c005de0 comp rx=0 tx=0).stop 2024-04-25T01:44:52.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.550+0000 7fb441528700 1 -- 172.21.15.78:0/3963960176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb43c1012c0 msgr2=0x7fb43c1a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:52.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.550+0000 7fb441528700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb43c1012c0 0x7fb43c1a0d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb42400b530 tx=0x7fb42400b8f0 comp rx=0 tx=0).stop 2024-04-25T01:44:52.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.550+0000 7fb441528700 1 -- 172.21.15.78:0/3963960176 shutdown_connections 2024-04-25T01:44:52.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.550+0000 7fb441528700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb4280a2dc0 0x7fb4280a5280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:52.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.550+0000 7fb441528700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43c103140 0x7fb43c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:52.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.550+0000 7fb441528700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb43c100820 0x7fb43c1a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:52.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.550+0000 7fb441528700 1 --2- 172.21.15.78:0/3963960176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb43c1012c0 0x7fb43c1a0d80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:52.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.550+0000 7fb441528700 1 -- 172.21.15.78:0/3963960176 >> 172.21.15.78:0/3963960176 conn(0x7fb43c0777a0 msgr2=0x7fb43c0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:52.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.550+0000 7fb441528700 1 -- 172.21.15.78:0/3963960176 shutdown_connections 2024-04-25T01:44:52.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:52.550+0000 7fb441528700 1 -- 172.21.15.78:0/3963960176 wait complete. 2024-04-25T01:44:52.905 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299439 got 223338299444 for osd.7 2024-04-25T01:44:52.905 DEBUG:teuthology.parallel:result is None 2024-04-25T01:44:53.215 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:52 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3963960176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T01:44:53.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:52 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3963960176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T01:44:53.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:52 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3963960176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T01:44:53.551 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411406 for osd.2 2024-04-25T01:44:53.551 DEBUG:teuthology.parallel:result is None 2024-04-25T01:44:53.551 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T01:44:53.552 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T01:44:53.812 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:54.075 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:53 smithi078 ceph-mon[29469]: pgmap v340: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-25T01:44:54.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:53 smithi152 ceph-mon[32465]: pgmap v340: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-25T01:44:54.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:53 smithi196 ceph-mon[32978]: pgmap v340: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-25T01:44:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.629+0000 7faf366e4700 1 -- 172.21.15.78:0/430972862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf30107570 msgr2=0x7faf3010f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.629+0000 7faf366e4700 1 --2- 172.21.15.78:0/430972862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf30107570 0x7faf3010f140 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf2800b0a0 tx=0x7faf2800b3b0 comp rx=0 tx=0).stop 2024-04-25T01:44:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.629+0000 7faf366e4700 1 -- 172.21.15.78:0/430972862 shutdown_connections 2024-04-25T01:44:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.629+0000 7faf366e4700 1 --2- 172.21.15.78:0/430972862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf30107570 0x7faf3010f140 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.629+0000 7faf366e4700 1 --2- 172.21.15.78:0/430972862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf301024e0 0x7faf301028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:55.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.629+0000 7faf366e4700 1 --2- 172.21.15.78:0/430972862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf30102e90 0x7faf30106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:55.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.629+0000 7faf366e4700 1 -- 172.21.15.78:0/430972862 >> 172.21.15.78:0/430972862 conn(0x7faf30079390 msgr2=0x7faf300797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:55.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.629+0000 7faf366e4700 1 -- 172.21.15.78:0/430972862 shutdown_connections 2024-04-25T01:44:55.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.629+0000 7faf366e4700 1 -- 172.21.15.78:0/430972862 wait complete. 2024-04-25T01:44:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.630+0000 7faf366e4700 1 Processor -- start 2024-04-25T01:44:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.630+0000 7faf366e4700 1 -- start start 2024-04-25T01:44:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.630+0000 7faf366e4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf301024e0 0x7faf3019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.630+0000 7faf366e4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf30102e90 0x7faf3019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.630+0000 7faf366e4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf30107570 0x7faf30198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.630+0000 7faf366e4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf30112550 con 0x7faf301024e0 2024-04-25T01:44:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.630+0000 7faf366e4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faf301123d0 con 0x7faf30102e90 2024-04-25T01:44:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.630+0000 7faf366e4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faf301126d0 con 0x7faf30107570 2024-04-25T01:44:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.630+0000 7faf2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf301024e0 0x7faf3019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.630+0000 7faf2ffff700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf301024e0 0x7faf3019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37424/0 (socket says 172.21.15.78:37424) 2024-04-25T01:44:55.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.630+0000 7faf2ffff700 1 -- 172.21.15.78:0/1011033120 learned_addr learned my addr 172.21.15.78:0/1011033120 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.631+0000 7faf2f7fe700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf30102e90 0x7faf3019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.631+0000 7faf34c81700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf30107570 0x7faf30198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.631+0000 7faf2ffff700 1 -- 172.21.15.78:0/1011033120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf30107570 msgr2=0x7faf30198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.631+0000 7faf2ffff700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf30107570 0x7faf30198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.631+0000 7faf2ffff700 1 -- 172.21.15.78:0/1011033120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf30102e90 msgr2=0x7faf3019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.631+0000 7faf2ffff700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf30102e90 0x7faf3019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.631+0000 7faf2ffff700 1 -- 172.21.15.78:0/1011033120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf30199050 con 0x7faf301024e0 2024-04-25T01:44:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.631+0000 7faf34c81700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf30107570 0x7faf30198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:44:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.631+0000 7faf2ffff700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf301024e0 0x7faf3019e6a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faf2000c9b0 tx=0x7faf2000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.631+0000 7faf2d7fa700 1 -- 172.21.15.78:0/1011033120 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20007b00 con 0x7faf301024e0 2024-04-25T01:44:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.631+0000 7faf366e4700 1 -- 172.21.15.78:0/1011033120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf30199340 con 0x7faf301024e0 2024-04-25T01:44:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.631+0000 7faf2d7fa700 1 -- 172.21.15.78:0/1011033120 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf20007ca0 con 0x7faf301024e0 2024-04-25T01:44:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.631+0000 7faf2d7fa700 1 -- 172.21.15.78:0/1011033120 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20018730 con 0x7faf301024e0 2024-04-25T01:44:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.631+0000 7faf366e4700 1 -- 172.21.15.78:0/1011033120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf301a5620 con 0x7faf301024e0 2024-04-25T01:44:55.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.632+0000 7faf366e4700 1 -- 172.21.15.78:0/1011033120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf30103970 con 0x7faf301024e0 2024-04-25T01:44:55.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.633+0000 7faf2d7fa700 1 -- 172.21.15.78:0/1011033120 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faf200188d0 con 0x7faf301024e0 2024-04-25T01:44:55.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.634+0000 7faf2d7fa700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7faf180a2c20 0x7faf180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:55.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.634+0000 7faf2f7fe700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7faf180a2c20 0x7faf180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:55.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.634+0000 7faf2d7fa700 1 -- 172.21.15.78:0/1011033120 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7faf200cf2f0 con 0x7faf301024e0 2024-04-25T01:44:55.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.634+0000 7faf2f7fe700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7faf180a2c20 0x7faf180a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faf30199f30 tx=0x7faf2400a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:55.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.638+0000 7faf2d7fa700 1 -- 172.21.15.78:0/1011033120 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faf20014030 con 0x7faf301024e0 2024-04-25T01:44:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.831+0000 7faf366e4700 1 -- 172.21.15.78:0/1011033120 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf30199ca0 con 0x7faf180a2c20 2024-04-25T01:44:55.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.840+0000 7faf2d7fa700 1 -- 172.21.15.78:0/1011033120 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340861 (secure 0 0 0) 0x7faf30199ca0 con 0x7faf180a2c20 2024-04-25T01:44:55.838 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:44:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.843+0000 7faf366e4700 1 -- 172.21.15.78:0/1011033120 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7faf180a2c20 msgr2=0x7faf180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.843+0000 7faf366e4700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7faf180a2c20 0x7faf180a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faf30199f30 tx=0x7faf2400a300 comp rx=0 tx=0).stop 2024-04-25T01:44:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.843+0000 7faf366e4700 1 -- 172.21.15.78:0/1011033120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf301024e0 msgr2=0x7faf3019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.843+0000 7faf366e4700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf301024e0 0x7faf3019e6a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faf2000c9b0 tx=0x7faf2000ccc0 comp rx=0 tx=0).stop 2024-04-25T01:44:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.843+0000 7faf366e4700 1 -- 172.21.15.78:0/1011033120 shutdown_connections 2024-04-25T01:44:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.843+0000 7faf366e4700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7faf180a2c20 0x7faf180a50e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.843+0000 7faf366e4700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf30107570 0x7faf30198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.843+0000 7faf366e4700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf301024e0 0x7faf3019e6a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.843+0000 7faf366e4700 1 --2- 172.21.15.78:0/1011033120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf30102e90 0x7faf3019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.843+0000 7faf366e4700 1 -- 172.21.15.78:0/1011033120 >> 172.21.15.78:0/1011033120 conn(0x7faf30079390 msgr2=0x7faf30100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.844+0000 7faf366e4700 1 -- 172.21.15.78:0/1011033120 shutdown_connections 2024-04-25T01:44:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:55.844+0000 7faf366e4700 1 -- 172.21.15.78:0/1011033120 wait complete. 2024-04-25T01:44:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-04-25T01:44:56.149 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:55 smithi078 ceph-mon[29469]: pgmap v341: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-25T01:44:56.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:55 smithi152 ceph-mon[32465]: pgmap v341: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-25T01:44:56.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:55 smithi196 ceph-mon[32978]: pgmap v341: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-25T01:44:56.698 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-04-25T01:44:54.918537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662225,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":871,"num_read_kb":559,"num_write":436,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":860,"ondisk_log_size":860,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361592,"kb_used_data":12440,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580232,"statfs":{"total":1151940427776,"available":1151570157568,"internally_reserved":0,"allocated":12738560,"data_stored":8025749,"data_compressed":360162,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":59,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003662"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.935647+0000","last_change":"2024-04-25T01:43:17.423128+0000","last_active":"2024-04-25T01:44:46.935647+0000","last_peered":"2024-04-25T01:44:46.935647+0000","last_clean":"2024-04-25T01:44:46.935647+0000","last_became_active":"2024-04-25T01:43:17.422356+0000","last_became_peered":"2024-04-25T01:43:17.422356+0000","last_unstale":"2024-04-25T01:44:46.935647+0000","last_undegraded":"2024-04-25T01:44:46.935647+0000","last_fullsized":"2024-04-25T01:44:46.935647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:05.524727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.910330+0000","last_change":"2024-04-25T01:43:17.422866+0000","last_active":"2024-04-25T01:44:46.910330+0000","last_peered":"2024-04-25T01:44:46.910330+0000","last_clean":"2024-04-25T01:44:46.910330+0000","last_became_active":"2024-04-25T01:43:17.422574+0000","last_became_peered":"2024-04-25T01:43:17.422574+0000","last_unstale":"2024-04-25T01:44:46.910330+0000","last_undegraded":"2024-04-25T01:44:46.910330+0000","last_fullsized":"2024-04-25T01:44:46.910330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.842361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919618+0000","last_change":"2024-04-25T01:43:17.419320+0000","last_active":"2024-04-25T01:44:46.919618+0000","last_peered":"2024-04-25T01:44:46.919618+0000","last_clean":"2024-04-25T01:44:46.919618+0000","last_became_active":"2024-04-25T01:43:17.418988+0000","last_became_peered":"2024-04-25T01:43:17.418988+0000","last_unstale":"2024-04-25T01:44:46.919618+0000","last_undegraded":"2024-04-25T01:44:46.919618+0000","last_fullsized":"2024-04-25T01:44:46.919618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:18.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.921155+0000","last_change":"2024-04-25T01:43:17.430061+0000","last_active":"2024-04-25T01:44:46.921155+0000","last_peered":"2024-04-25T01:44:46.921155+0000","last_clean":"2024-04-25T01:44:46.921155+0000","last_became_active":"2024-04-25T01:43:17.429736+0000","last_became_peered":"2024-04-25T01:43:17.429736+0000","last_unstale":"2024-04-25T01:44:46.921155+0000","last_undegraded":"2024-04-25T01:44:46.921155+0000","last_fullsized":"2024-04-25T01:44:46.921155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:57.784221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.915675+0000","last_change":"2024-04-25T01:43:17.422876+0000","last_active":"2024-04-25T01:44:46.915675+0000","last_peered":"2024-04-25T01:44:46.915675+0000","last_clean":"2024-04-25T01:44:46.915675+0000","last_became_active":"2024-04-25T01:43:17.422523+0000","last_became_peered":"2024-04-25T01:43:17.422523+0000","last_unstale":"2024-04-25T01:44:46.915675+0000","last_undegraded":"2024-04-25T01:44:46.915675+0000","last_fullsized":"2024-04-25T01:44:46.915675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:14.328021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919150+0000","last_change":"2024-04-25T01:43:17.424941+0000","last_active":"2024-04-25T01:44:46.919150+0000","last_peered":"2024-04-25T01:44:46.919150+0000","last_clean":"2024-04-25T01:44:46.919150+0000","last_became_active":"2024-04-25T01:43:17.424620+0000","last_became_peered":"2024-04-25T01:43:17.424620+0000","last_unstale":"2024-04-25T01:44:46.919150+0000","last_undegraded":"2024-04-25T01:44:46.919150+0000","last_fullsized":"2024-04-25T01:44:46.919150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:06.111590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.917177+0000","last_change":"2024-04-25T01:43:17.429188+0000","last_active":"2024-04-25T01:44:46.917177+0000","last_peered":"2024-04-25T01:44:46.917177+0000","last_clean":"2024-04-25T01:44:46.917177+0000","last_became_active":"2024-04-25T01:43:17.428837+0000","last_became_peered":"2024-04-25T01:43:17.428837+0000","last_unstale":"2024-04-25T01:44:46.917177+0000","last_undegraded":"2024-04-25T01:44:46.917177+0000","last_fullsized":"2024-04-25T01:44:46.917177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:08.923208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.918468+0000","last_change":"2024-04-25T01:43:17.431402+0000","last_active":"2024-04-25T01:44:46.918468+0000","last_peered":"2024-04-25T01:44:46.918468+0000","last_clean":"2024-04-25T01:44:46.918468+0000","last_became_active":"2024-04-25T01:43:17.431044+0000","last_became_peered":"2024-04-25T01:43:17.431044+0000","last_unstale":"2024-04-25T01:44:46.918468+0000","last_undegraded":"2024-04-25T01:44:46.918468+0000","last_fullsized":"2024-04-25T01:44:46.918468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:20.731589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.937188+0000","last_change":"2024-04-25T01:43:17.430029+0000","last_active":"2024-04-25T01:44:46.937188+0000","last_peered":"2024-04-25T01:44:46.937188+0000","last_clean":"2024-04-25T01:44:46.937188+0000","last_became_active":"2024-04-25T01:43:17.428588+0000","last_became_peered":"2024-04-25T01:43:17.428588+0000","last_unstale":"2024-04-25T01:44:46.937188+0000","last_undegraded":"2024-04-25T01:44:46.937188+0000","last_fullsized":"2024-04-25T01:44:46.937188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:58.520175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.920958+0000","last_change":"2024-04-25T01:43:17.430377+0000","last_active":"2024-04-25T01:44:46.920958+0000","last_peered":"2024-04-25T01:44:46.920958+0000","last_clean":"2024-04-25T01:44:46.920958+0000","last_became_active":"2024-04-25T01:43:17.426609+0000","last_became_peered":"2024-04-25T01:43:17.426609+0000","last_unstale":"2024-04-25T01:44:46.920958+0000","last_undegraded":"2024-04-25T01:44:46.920958+0000","last_fullsized":"2024-04-25T01:44:46.920958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:10.696040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.936196+0000","last_change":"2024-04-25T01:43:17.431561+0000","last_active":"2024-04-25T01:44:46.936196+0000","last_peered":"2024-04-25T01:44:46.936196+0000","last_clean":"2024-04-25T01:44:46.936196+0000","last_became_active":"2024-04-25T01:43:17.431229+0000","last_became_peered":"2024-04-25T01:43:17.431229+0000","last_unstale":"2024-04-25T01:44:46.936196+0000","last_undegraded":"2024-04-25T01:44:46.936196+0000","last_fullsized":"2024-04-25T01:44:46.936196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:03.686373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.923858+0000","last_change":"2024-04-25T01:43:17.428442+0000","last_active":"2024-04-25T01:44:46.923858+0000","last_peered":"2024-04-25T01:44:46.923858+0000","last_clean":"2024-04-25T01:44:46.923858+0000","last_became_active":"2024-04-25T01:43:17.428052+0000","last_became_peered":"2024-04-25T01:43:17.428052+0000","last_unstale":"2024-04-25T01:44:46.923858+0000","last_undegraded":"2024-04-25T01:44:46.923858+0000","last_fullsized":"2024-04-25T01:44:46.923858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:06.097229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.913370+0000","last_change":"2024-04-25T01:43:17.420520+0000","last_active":"2024-04-25T01:44:46.913370+0000","last_peered":"2024-04-25T01:44:46.913370+0000","last_clean":"2024-04-25T01:44:46.913370+0000","last_became_active":"2024-04-25T01:43:17.420171+0000","last_became_peered":"2024-04-25T01:43:17.420171+0000","last_unstale":"2024-04-25T01:44:46.913370+0000","last_undegraded":"2024-04-25T01:44:46.913370+0000","last_fullsized":"2024-04-25T01:44:46.913370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:43.699472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.920581+0000","last_change":"2024-04-25T01:43:17.422114+0000","last_active":"2024-04-25T01:44:46.920581+0000","last_peered":"2024-04-25T01:44:46.920581+0000","last_clean":"2024-04-25T01:44:46.920581+0000","last_became_active":"2024-04-25T01:43:17.421729+0000","last_became_peered":"2024-04-25T01:43:17.421729+0000","last_unstale":"2024-04-25T01:44:46.920581+0000","last_undegraded":"2024-04-25T01:44:46.920581+0000","last_fullsized":"2024-04-25T01:44:46.920581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:00.955123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919833+0000","last_change":"2024-04-25T01:43:17.427344+0000","last_active":"2024-04-25T01:44:46.919833+0000","last_peered":"2024-04-25T01:44:46.919833+0000","last_clean":"2024-04-25T01:44:46.919833+0000","last_became_active":"2024-04-25T01:43:17.427044+0000","last_became_peered":"2024-04-25T01:43:17.427044+0000","last_unstale":"2024-04-25T01:44:46.919833+0000","last_undegraded":"2024-04-25T01:44:46.919833+0000","last_fullsized":"2024-04-25T01:44:46.919833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:24.564654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.933157+0000","last_change":"2024-04-25T01:43:17.427604+0000","last_active":"2024-04-25T01:44:46.933157+0000","last_peered":"2024-04-25T01:44:46.933157+0000","last_clean":"2024-04-25T01:44:46.933157+0000","last_became_active":"2024-04-25T01:43:17.427286+0000","last_became_peered":"2024-04-25T01:43:17.427286+0000","last_unstale":"2024-04-25T01:44:46.933157+0000","last_undegraded":"2024-04-25T01:44:46.933157+0000","last_fullsized":"2024-04-25T01:44:46.933157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:25.584150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.938412+0000","last_change":"2024-04-25T01:43:17.423401+0000","last_active":"2024-04-25T01:44:46.938412+0000","last_peered":"2024-04-25T01:44:46.938412+0000","last_clean":"2024-04-25T01:44:46.938412+0000","last_became_active":"2024-04-25T01:43:17.422807+0000","last_became_peered":"2024-04-25T01:43:17.422807+0000","last_unstale":"2024-04-25T01:44:46.938412+0000","last_undegraded":"2024-04-25T01:44:46.938412+0000","last_fullsized":"2024-04-25T01:44:46.938412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:58.461242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.935691+0000","last_change":"2024-04-25T01:43:17.422054+0000","last_active":"2024-04-25T01:44:46.935691+0000","last_peered":"2024-04-25T01:44:46.935691+0000","last_clean":"2024-04-25T01:44:46.935691+0000","last_became_active":"2024-04-25T01:43:17.421746+0000","last_became_peered":"2024-04-25T01:43:17.421746+0000","last_unstale":"2024-04-25T01:44:46.935691+0000","last_undegraded":"2024-04-25T01:44:46.935691+0000","last_fullsized":"2024-04-25T01:44:46.935691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:28.405852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.916459+0000","last_change":"2024-04-25T01:43:17.429288+0000","last_active":"2024-04-25T01:44:46.916459+0000","last_peered":"2024-04-25T01:44:46.916459+0000","last_clean":"2024-04-25T01:44:46.916459+0000","last_became_active":"2024-04-25T01:43:17.427794+0000","last_became_peered":"2024-04-25T01:43:17.427794+0000","last_unstale":"2024-04-25T01:44:46.916459+0000","last_undegraded":"2024-04-25T01:44:46.916459+0000","last_fullsized":"2024-04-25T01:44:46.916459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:33.003361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.515428+0000","last_change":"2024-04-25T01:43:21.438427+0000","last_active":"2024-04-25T01:44:18.515428+0000","last_peered":"2024-04-25T01:44:18.515428+0000","last_clean":"2024-04-25T01:44:18.515428+0000","last_became_active":"2024-04-25T01:43:21.438018+0000","last_became_peered":"2024-04-25T01:43:21.438018+0000","last_unstale":"2024-04-25T01:44:18.515428+0000","last_undegraded":"2024-04-25T01:44:18.515428+0000","last_fullsized":"2024-04-25T01:44:18.515428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:46.509437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":109,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:50.530861+0000","last_change":"2024-04-25T01:43:17.031110+0000","last_active":"2024-04-25T01:44:50.530861+0000","last_peered":"2024-04-25T01:44:50.530861+0000","last_clean":"2024-04-25T01:44:50.530861+0000","last_became_active":"2024-04-25T01:43:17.030806+0000","last_became_peered":"2024-04-25T01:43:17.030806+0000","last_unstale":"2024-04-25T01:44:50.530861+0000","last_undegraded":"2024-04-25T01:44:50.530861+0000","last_fullsized":"2024-04-25T01:44:50.530861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:52:12.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.917790+0000","last_change":"2024-04-25T01:43:17.428847+0000","last_active":"2024-04-25T01:44:46.917790+0000","last_peered":"2024-04-25T01:44:46.917790+0000","last_clean":"2024-04-25T01:44:46.917790+0000","last_became_active":"2024-04-25T01:43:17.428553+0000","last_became_peered":"2024-04-25T01:43:17.428553+0000","last_unstale":"2024-04-25T01:44:46.917790+0000","last_undegraded":"2024-04-25T01:44:46.917790+0000","last_fullsized":"2024-04-25T01:44:46.917790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:13.973275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.582673+0000","last_change":"2024-04-25T01:44:18.582673+0000","last_active":"2024-04-25T01:44:18.582673+0000","last_peered":"2024-04-25T01:44:18.582673+0000","last_clean":"2024-04-25T01:44:18.582673+0000","last_became_active":"2024-04-25T01:44:18.582312+0000","last_became_peered":"2024-04-25T01:44:18.582312+0000","last_unstale":"2024-04-25T01:44:18.582673+0000","last_undegraded":"2024-04-25T01:44:18.582673+0000","last_fullsized":"2024-04-25T01:44:18.582673+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:22.909294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.581774+0000","last_change":"2024-04-25T01:43:17.032954+0000","last_active":"2024-04-25T01:44:18.581774+0000","last_peered":"2024-04-25T01:44:18.581774+0000","last_clean":"2024-04-25T01:44:18.581774+0000","last_became_active":"2024-04-25T01:43:17.032627+0000","last_became_peered":"2024-04-25T01:43:17.032627+0000","last_unstale":"2024-04-25T01:44:18.581774+0000","last_undegraded":"2024-04-25T01:44:18.581774+0000","last_fullsized":"2024-04-25T01:44:18.581774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:07.167227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.917301+0000","last_change":"2024-04-25T01:43:17.419451+0000","last_active":"2024-04-25T01:44:46.917301+0000","last_peered":"2024-04-25T01:44:46.917301+0000","last_clean":"2024-04-25T01:44:46.917301+0000","last_became_active":"2024-04-25T01:43:17.419111+0000","last_became_peered":"2024-04-25T01:43:17.419111+0000","last_unstale":"2024-04-25T01:44:46.917301+0000","last_undegraded":"2024-04-25T01:44:46.917301+0000","last_fullsized":"2024-04-25T01:44:46.917301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:41.284248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512163+0000","last_change":"2024-04-25T01:43:21.452525+0000","last_active":"2024-04-25T01:44:18.512163+0000","last_peered":"2024-04-25T01:44:18.512163+0000","last_clean":"2024-04-25T01:44:18.512163+0000","last_became_active":"2024-04-25T01:43:21.451834+0000","last_became_peered":"2024-04-25T01:43:21.451834+0000","last_unstale":"2024-04-25T01:44:18.512163+0000","last_undegraded":"2024-04-25T01:44:18.512163+0000","last_fullsized":"2024-04-25T01:44:18.512163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:34.866314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.510174+0000","last_change":"2024-04-25T01:43:17.035737+0000","last_active":"2024-04-25T01:44:18.510174+0000","last_peered":"2024-04-25T01:44:18.510174+0000","last_clean":"2024-04-25T01:44:18.510174+0000","last_became_active":"2024-04-25T01:43:17.034709+0000","last_became_peered":"2024-04-25T01:43:17.034709+0000","last_unstale":"2024-04-25T01:44:18.510174+0000","last_undegraded":"2024-04-25T01:44:18.510174+0000","last_fullsized":"2024-04-25T01:44:18.510174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:50.419193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.936816+0000","last_change":"2024-04-25T01:43:17.422530+0000","last_active":"2024-04-25T01:44:46.936816+0000","last_peered":"2024-04-25T01:44:46.936816+0000","last_clean":"2024-04-25T01:44:46.936816+0000","last_became_active":"2024-04-25T01:43:17.422197+0000","last_became_peered":"2024-04-25T01:43:17.422197+0000","last_unstale":"2024-04-25T01:44:46.936816+0000","last_undegraded":"2024-04-25T01:44:46.936816+0000","last_fullsized":"2024-04-25T01:44:46.936816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:25.250947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511486+0000","last_change":"2024-04-25T01:43:21.446869+0000","last_active":"2024-04-25T01:44:18.511486+0000","last_peered":"2024-04-25T01:44:18.511486+0000","last_clean":"2024-04-25T01:44:18.511486+0000","last_became_active":"2024-04-25T01:43:21.446446+0000","last_became_peered":"2024-04-25T01:43:21.446446+0000","last_unstale":"2024-04-25T01:44:18.511486+0000","last_undegraded":"2024-04-25T01:44:18.511486+0000","last_fullsized":"2024-04-25T01:44:18.511486+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:21.023845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.510177+0000","last_change":"2024-04-25T01:43:17.027614+0000","last_active":"2024-04-25T01:44:18.510177+0000","last_peered":"2024-04-25T01:44:18.510177+0000","last_clean":"2024-04-25T01:44:18.510177+0000","last_became_active":"2024-04-25T01:43:17.027235+0000","last_became_peered":"2024-04-25T01:43:17.027235+0000","last_unstale":"2024-04-25T01:44:18.510177+0000","last_undegraded":"2024-04-25T01:44:18.510177+0000","last_fullsized":"2024-04-25T01:44:18.510177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:45.627146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.917903+0000","last_change":"2024-04-25T01:43:17.425813+0000","last_active":"2024-04-25T01:44:46.917903+0000","last_peered":"2024-04-25T01:44:46.917903+0000","last_clean":"2024-04-25T01:44:46.917903+0000","last_became_active":"2024-04-25T01:43:17.425469+0000","last_became_peered":"2024-04-25T01:43:17.425469+0000","last_unstale":"2024-04-25T01:44:46.917903+0000","last_undegraded":"2024-04-25T01:44:46.917903+0000","last_fullsized":"2024-04-25T01:44:46.917903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:49.863358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.513249+0000","last_change":"2024-04-25T01:43:21.444557+0000","last_active":"2024-04-25T01:44:18.513249+0000","last_peered":"2024-04-25T01:44:18.513249+0000","last_clean":"2024-04-25T01:44:18.513249+0000","last_became_active":"2024-04-25T01:43:21.443809+0000","last_became_peered":"2024-04-25T01:43:21.443809+0000","last_unstale":"2024-04-25T01:44:18.513249+0000","last_undegraded":"2024-04-25T01:44:18.513249+0000","last_fullsized":"2024-04-25T01:44:18.513249+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:29.488448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":110,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:52.523424+0000","last_change":"2024-04-25T01:43:17.032280+0000","last_active":"2024-04-25T01:44:52.523424+0000","last_peered":"2024-04-25T01:44:52.523424+0000","last_clean":"2024-04-25T01:44:52.523424+0000","last_became_active":"2024-04-25T01:43:17.031649+0000","last_became_peered":"2024-04-25T01:43:17.031649+0000","last_unstale":"2024-04-25T01:44:52.523424+0000","last_undegraded":"2024-04-25T01:44:52.523424+0000","last_fullsized":"2024-04-25T01:44:52.523424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:08.532261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.914735+0000","last_change":"2024-04-25T01:44:18.514557+0000","last_active":"2024-04-25T01:44:46.914735+0000","last_peered":"2024-04-25T01:44:46.914735+0000","last_clean":"2024-04-25T01:44:46.914735+0000","last_became_active":"2024-04-25T01:44:18.514194+0000","last_became_peered":"2024-04-25T01:44:18.514194+0000","last_unstale":"2024-04-25T01:44:46.914735+0000","last_undegraded":"2024-04-25T01:44:46.914735+0000","last_fullsized":"2024-04-25T01:44:46.914735+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:43.110217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.513117+0000","last_change":"2024-04-25T01:43:21.442683+0000","last_active":"2024-04-25T01:44:18.513117+0000","last_peered":"2024-04-25T01:44:18.513117+0000","last_clean":"2024-04-25T01:44:18.513117+0000","last_became_active":"2024-04-25T01:43:21.442222+0000","last_became_peered":"2024-04-25T01:43:21.442222+0000","last_unstale":"2024-04-25T01:44:18.513117+0000","last_undegraded":"2024-04-25T01:44:18.513117+0000","last_fullsized":"2024-04-25T01:44:18.513117+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:25.196600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.510042+0000","last_change":"2024-04-25T01:43:17.028054+0000","last_active":"2024-04-25T01:44:18.510042+0000","last_peered":"2024-04-25T01:44:18.510042+0000","last_clean":"2024-04-25T01:44:18.510042+0000","last_became_active":"2024-04-25T01:43:17.027639+0000","last_became_peered":"2024-04-25T01:43:17.027639+0000","last_unstale":"2024-04-25T01:44:18.510042+0000","last_undegraded":"2024-04-25T01:44:18.510042+0000","last_fullsized":"2024-04-25T01:44:18.510042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:40.569221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.924130+0000","last_change":"2024-04-25T01:43:17.430417+0000","last_active":"2024-04-25T01:44:46.924130+0000","last_peered":"2024-04-25T01:44:46.924130+0000","last_clean":"2024-04-25T01:44:46.924130+0000","last_became_active":"2024-04-25T01:43:17.430024+0000","last_became_peered":"2024-04-25T01:43:17.430024+0000","last_unstale":"2024-04-25T01:44:46.924130+0000","last_undegraded":"2024-04-25T01:44:46.924130+0000","last_fullsized":"2024-04-25T01:44:46.924130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:56.834530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511592+0000","last_change":"2024-04-25T01:43:21.447732+0000","last_active":"2024-04-25T01:44:18.511592+0000","last_peered":"2024-04-25T01:44:18.511592+0000","last_clean":"2024-04-25T01:44:18.511592+0000","last_became_active":"2024-04-25T01:43:21.447315+0000","last_became_peered":"2024-04-25T01:43:21.447315+0000","last_unstale":"2024-04-25T01:44:18.511592+0000","last_undegraded":"2024-04-25T01:44:18.511592+0000","last_fullsized":"2024-04-25T01:44:18.511592+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:18.138828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511674+0000","last_change":"2024-04-25T01:43:17.018769+0000","last_active":"2024-04-25T01:44:18.511674+0000","last_peered":"2024-04-25T01:44:18.511674+0000","last_clean":"2024-04-25T01:44:18.511674+0000","last_became_active":"2024-04-25T01:43:17.018169+0000","last_became_peered":"2024-04-25T01:43:17.018169+0000","last_unstale":"2024-04-25T01:44:18.511674+0000","last_undegraded":"2024-04-25T01:44:18.511674+0000","last_fullsized":"2024-04-25T01:44:18.511674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:59.112199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.915017+0000","last_change":"2024-04-25T01:43:17.426289+0000","last_active":"2024-04-25T01:44:46.915017+0000","last_peered":"2024-04-25T01:44:46.915017+0000","last_clean":"2024-04-25T01:44:46.915017+0000","last_became_active":"2024-04-25T01:43:17.425512+0000","last_became_peered":"2024-04-25T01:43:17.425512+0000","last_unstale":"2024-04-25T01:44:46.915017+0000","last_undegraded":"2024-04-25T01:44:46.915017+0000","last_fullsized":"2024-04-25T01:44:46.915017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:47.956758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512154+0000","last_change":"2024-04-25T01:43:21.449131+0000","last_active":"2024-04-25T01:44:18.512154+0000","last_peered":"2024-04-25T01:44:18.512154+0000","last_clean":"2024-04-25T01:44:18.512154+0000","last_became_active":"2024-04-25T01:43:21.448752+0000","last_became_peered":"2024-04-25T01:43:21.448752+0000","last_unstale":"2024-04-25T01:44:18.512154+0000","last_undegraded":"2024-04-25T01:44:18.512154+0000","last_fullsized":"2024-04-25T01:44:18.512154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:23.109582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.514072+0000","last_change":"2024-04-25T01:43:17.032414+0000","last_active":"2024-04-25T01:44:18.514072+0000","last_peered":"2024-04-25T01:44:18.514072+0000","last_clean":"2024-04-25T01:44:18.514072+0000","last_became_active":"2024-04-25T01:43:17.031848+0000","last_became_peered":"2024-04-25T01:43:17.031848+0000","last_unstale":"2024-04-25T01:44:18.514072+0000","last_undegraded":"2024-04-25T01:44:18.514072+0000","last_fullsized":"2024-04-25T01:44:18.514072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:37.034113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.917038+0000","last_change":"2024-04-25T01:43:17.422831+0000","last_active":"2024-04-25T01:44:46.917038+0000","last_peered":"2024-04-25T01:44:46.917038+0000","last_clean":"2024-04-25T01:44:46.917038+0000","last_became_active":"2024-04-25T01:43:17.422238+0000","last_became_peered":"2024-04-25T01:43:17.422238+0000","last_unstale":"2024-04-25T01:44:46.917038+0000","last_undegraded":"2024-04-25T01:44:46.917038+0000","last_fullsized":"2024-04-25T01:44:46.917038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:05.740054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.580949+0000","last_change":"2024-04-25T01:43:21.442268+0000","last_active":"2024-04-25T01:44:18.580949+0000","last_peered":"2024-04-25T01:44:18.580949+0000","last_clean":"2024-04-25T01:44:18.580949+0000","last_became_active":"2024-04-25T01:43:21.441844+0000","last_became_peered":"2024-04-25T01:43:21.441844+0000","last_unstale":"2024-04-25T01:44:18.580949+0000","last_undegraded":"2024-04-25T01:44:18.580949+0000","last_fullsized":"2024-04-25T01:44:18.580949+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:21.627312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'6","reported_seq":171,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:51.521233+0000","last_change":"2024-04-25T01:43:17.022576+0000","last_active":"2024-04-25T01:44:51.521233+0000","last_peered":"2024-04-25T01:44:51.521233+0000","last_clean":"2024-04-25T01:44:51.521233+0000","last_became_active":"2024-04-25T01:43:17.022223+0000","last_became_peered":"2024-04-25T01:43:17.022223+0000","last_unstale":"2024-04-25T01:44:51.521233+0000","last_undegraded":"2024-04-25T01:44:51.521233+0000","last_fullsized":"2024-04-25T01:44:51.521233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:23.905397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.937347+0000","last_change":"2024-04-25T01:43:17.421460+0000","last_active":"2024-04-25T01:44:46.937347+0000","last_peered":"2024-04-25T01:44:46.937347+0000","last_clean":"2024-04-25T01:44:46.937347+0000","last_became_active":"2024-04-25T01:43:17.421153+0000","last_became_peered":"2024-04-25T01:43:17.421153+0000","last_unstale":"2024-04-25T01:44:46.937347+0000","last_undegraded":"2024-04-25T01:44:46.937347+0000","last_fullsized":"2024-04-25T01:44:46.937347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:50.285476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509118+0000","last_change":"2024-04-25T01:43:21.450961+0000","last_active":"2024-04-25T01:44:18.509118+0000","last_peered":"2024-04-25T01:44:18.509118+0000","last_clean":"2024-04-25T01:44:18.509118+0000","last_became_active":"2024-04-25T01:43:21.450578+0000","last_became_peered":"2024-04-25T01:43:21.450578+0000","last_unstale":"2024-04-25T01:44:18.509118+0000","last_undegraded":"2024-04-25T01:44:18.509118+0000","last_fullsized":"2024-04-25T01:44:18.509118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:04.222443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512764+0000","last_change":"2024-04-25T01:43:17.030973+0000","last_active":"2024-04-25T01:44:18.512764+0000","last_peered":"2024-04-25T01:44:18.512764+0000","last_clean":"2024-04-25T01:44:18.512764+0000","last_became_active":"2024-04-25T01:43:17.030458+0000","last_became_peered":"2024-04-25T01:43:17.030458+0000","last_unstale":"2024-04-25T01:44:18.512764+0000","last_undegraded":"2024-04-25T01:44:18.512764+0000","last_fullsized":"2024-04-25T01:44:18.512764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:19.187667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.921223+0000","last_change":"2024-04-25T01:44:18.517319+0000","last_active":"2024-04-25T01:44:46.921223+0000","last_peered":"2024-04-25T01:44:46.921223+0000","last_clean":"2024-04-25T01:44:46.921223+0000","last_became_active":"2024-04-25T01:44:18.517046+0000","last_became_peered":"2024-04-25T01:44:18.517046+0000","last_unstale":"2024-04-25T01:44:46.921223+0000","last_undegraded":"2024-04-25T01:44:46.921223+0000","last_fullsized":"2024-04-25T01:44:46.921223+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:10.066856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512411+0000","last_change":"2024-04-25T01:43:21.452287+0000","last_active":"2024-04-25T01:44:18.512411+0000","last_peered":"2024-04-25T01:44:18.512411+0000","last_clean":"2024-04-25T01:44:18.512411+0000","last_became_active":"2024-04-25T01:43:21.451765+0000","last_became_peered":"2024-04-25T01:43:21.451765+0000","last_unstale":"2024-04-25T01:44:18.512411+0000","last_undegraded":"2024-04-25T01:44:18.512411+0000","last_fullsized":"2024-04-25T01:44:18.512411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:02.353645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.515621+0000","last_change":"2024-04-25T01:43:17.022319+0000","last_active":"2024-04-25T01:44:18.515621+0000","last_peered":"2024-04-25T01:44:18.515621+0000","last_clean":"2024-04-25T01:44:18.515621+0000","last_became_active":"2024-04-25T01:43:17.021944+0000","last_became_peered":"2024-04-25T01:43:17.021944+0000","last_unstale":"2024-04-25T01:44:18.515621+0000","last_undegraded":"2024-04-25T01:44:18.515621+0000","last_fullsized":"2024-04-25T01:44:18.515621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:42.856997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.938103+0000","last_change":"2024-04-25T01:43:17.418164+0000","last_active":"2024-04-25T01:44:46.938103+0000","last_peered":"2024-04-25T01:44:46.938103+0000","last_clean":"2024-04-25T01:44:46.938103+0000","last_became_active":"2024-04-25T01:43:17.417846+0000","last_became_peered":"2024-04-25T01:43:17.417846+0000","last_unstale":"2024-04-25T01:44:46.938103+0000","last_undegraded":"2024-04-25T01:44:46.938103+0000","last_fullsized":"2024-04-25T01:44:46.938103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:25.973457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509285+0000","last_change":"2024-04-25T01:43:21.443429+0000","last_active":"2024-04-25T01:44:18.509285+0000","last_peered":"2024-04-25T01:44:18.509285+0000","last_clean":"2024-04-25T01:44:18.509285+0000","last_became_active":"2024-04-25T01:43:21.442210+0000","last_became_peered":"2024-04-25T01:43:21.442210+0000","last_unstale":"2024-04-25T01:44:18.509285+0000","last_undegraded":"2024-04-25T01:44:18.509285+0000","last_fullsized":"2024-04-25T01:44:18.509285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:29.578740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'37","reported_seq":213,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:51.530150+0000","last_change":"2024-04-25T01:43:17.020501+0000","last_active":"2024-04-25T01:44:51.530150+0000","last_peered":"2024-04-25T01:44:51.530150+0000","last_clean":"2024-04-25T01:44:51.530150+0000","last_became_active":"2024-04-25T01:43:17.020160+0000","last_became_peered":"2024-04-25T01:43:17.020160+0000","last_unstale":"2024-04-25T01:44:51.530150+0000","last_undegraded":"2024-04-25T01:44:51.530150+0000","last_fullsized":"2024-04-25T01:44:51.530150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:49.273381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":137,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.915575+0000","last_change":"2024-04-25T01:43:17.426895+0000","last_active":"2024-04-25T01:44:46.915575+0000","last_peered":"2024-04-25T01:44:46.915575+0000","last_clean":"2024-04-25T01:44:46.915575+0000","last_became_active":"2024-04-25T01:43:17.426572+0000","last_became_peered":"2024-04-25T01:43:17.426572+0000","last_unstale":"2024-04-25T01:44:46.915575+0000","last_undegraded":"2024-04-25T01:44:46.915575+0000","last_fullsized":"2024-04-25T01:44:46.915575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:36.166282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.515916+0000","last_change":"2024-04-25T01:43:21.442204+0000","last_active":"2024-04-25T01:44:18.515916+0000","last_peered":"2024-04-25T01:44:18.515916+0000","last_clean":"2024-04-25T01:44:18.515916+0000","last_became_active":"2024-04-25T01:43:21.441779+0000","last_became_peered":"2024-04-25T01:43:21.441779+0000","last_unstale":"2024-04-25T01:44:18.515916+0000","last_undegraded":"2024-04-25T01:44:18.515916+0000","last_fullsized":"2024-04-25T01:44:18.515916+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:12.075657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.513718+0000","last_change":"2024-04-25T01:43:17.028737+0000","last_active":"2024-04-25T01:44:18.513718+0000","last_peered":"2024-04-25T01:44:18.513718+0000","last_clean":"2024-04-25T01:44:18.513718+0000","last_became_active":"2024-04-25T01:43:17.028349+0000","last_became_peered":"2024-04-25T01:43:17.028349+0000","last_unstale":"2024-04-25T01:44:18.513718+0000","last_undegraded":"2024-04-25T01:44:18.513718+0000","last_fullsized":"2024-04-25T01:44:18.513718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:04.802232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.915856+0000","last_change":"2024-04-25T01:43:17.436315+0000","last_active":"2024-04-25T01:44:46.915856+0000","last_peered":"2024-04-25T01:44:46.915856+0000","last_clean":"2024-04-25T01:44:46.915856+0000","last_became_active":"2024-04-25T01:43:17.436007+0000","last_became_peered":"2024-04-25T01:43:17.436007+0000","last_unstale":"2024-04-25T01:44:46.915856+0000","last_undegraded":"2024-04-25T01:44:46.915856+0000","last_fullsized":"2024-04-25T01:44:46.915856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:49.091092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.518839+0000","last_change":"2024-04-25T01:44:18.518839+0000","last_active":"2024-04-25T01:44:18.518839+0000","last_peered":"2024-04-25T01:44:18.518839+0000","last_clean":"2024-04-25T01:44:18.518839+0000","last_became_active":"2024-04-25T01:44:18.518237+0000","last_became_peered":"2024-04-25T01:44:18.518237+0000","last_unstale":"2024-04-25T01:44:18.518839+0000","last_undegraded":"2024-04-25T01:44:18.518839+0000","last_fullsized":"2024-04-25T01:44:18.518839+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:51.963839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.513983+0000","last_change":"2024-04-25T01:43:17.029513+0000","last_active":"2024-04-25T01:44:18.513983+0000","last_peered":"2024-04-25T01:44:18.513983+0000","last_clean":"2024-04-25T01:44:18.513983+0000","last_became_active":"2024-04-25T01:43:17.029203+0000","last_became_peered":"2024-04-25T01:43:17.029203+0000","last_unstale":"2024-04-25T01:44:18.513983+0000","last_undegraded":"2024-04-25T01:44:18.513983+0000","last_fullsized":"2024-04-25T01:44:18.513983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:47.002587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.929759+0000","last_change":"2024-04-25T01:43:17.435273+0000","last_active":"2024-04-25T01:44:46.929759+0000","last_peered":"2024-04-25T01:44:46.929759+0000","last_clean":"2024-04-25T01:44:46.929759+0000","last_became_active":"2024-04-25T01:43:17.434892+0000","last_became_peered":"2024-04-25T01:43:17.434892+0000","last_unstale":"2024-04-25T01:44:46.929759+0000","last_undegraded":"2024-04-25T01:44:46.929759+0000","last_fullsized":"2024-04-25T01:44:46.929759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:55.342440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509673+0000","last_change":"2024-04-25T01:43:21.448363+0000","last_active":"2024-04-25T01:44:18.509673+0000","last_peered":"2024-04-25T01:44:18.509673+0000","last_clean":"2024-04-25T01:44:18.509673+0000","last_became_active":"2024-04-25T01:43:21.445980+0000","last_became_peered":"2024-04-25T01:43:21.445980+0000","last_unstale":"2024-04-25T01:44:18.509673+0000","last_undegraded":"2024-04-25T01:44:18.509673+0000","last_fullsized":"2024-04-25T01:44:18.509673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:05.890579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":110,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:52.532172+0000","last_change":"2024-04-25T01:43:17.028557+0000","last_active":"2024-04-25T01:44:52.532172+0000","last_peered":"2024-04-25T01:44:52.532172+0000","last_clean":"2024-04-25T01:44:52.532172+0000","last_became_active":"2024-04-25T01:43:17.028199+0000","last_became_peered":"2024-04-25T01:43:17.028199+0000","last_unstale":"2024-04-25T01:44:52.532172+0000","last_undegraded":"2024-04-25T01:44:52.532172+0000","last_fullsized":"2024-04-25T01:44:52.532172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:41.809220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.926494+0000","last_change":"2024-04-25T01:43:17.437002+0000","last_active":"2024-04-25T01:44:46.926494+0000","last_peered":"2024-04-25T01:44:46.926494+0000","last_clean":"2024-04-25T01:44:46.926494+0000","last_became_active":"2024-04-25T01:43:17.436509+0000","last_became_peered":"2024-04-25T01:43:17.436509+0000","last_unstale":"2024-04-25T01:44:46.926494+0000","last_undegraded":"2024-04-25T01:44:46.926494+0000","last_fullsized":"2024-04-25T01:44:46.926494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:49.329249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.582692+0000","last_change":"2024-04-25T01:44:18.582692+0000","last_active":"2024-04-25T01:44:18.582692+0000","last_peered":"2024-04-25T01:44:18.582692+0000","last_clean":"2024-04-25T01:44:18.582692+0000","last_became_active":"2024-04-25T01:44:18.582271+0000","last_became_peered":"2024-04-25T01:44:18.582271+0000","last_unstale":"2024-04-25T01:44:18.582692+0000","last_undegraded":"2024-04-25T01:44:18.582692+0000","last_fullsized":"2024-04-25T01:44:18.582692+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:56.946759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509402+0000","last_change":"2024-04-25T01:43:17.028185+0000","last_active":"2024-04-25T01:44:18.509402+0000","last_peered":"2024-04-25T01:44:18.509402+0000","last_clean":"2024-04-25T01:44:18.509402+0000","last_became_active":"2024-04-25T01:43:17.027851+0000","last_became_peered":"2024-04-25T01:43:17.027851+0000","last_unstale":"2024-04-25T01:44:18.509402+0000","last_undegraded":"2024-04-25T01:44:18.509402+0000","last_fullsized":"2024-04-25T01:44:18.509402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:02.320793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.918096+0000","last_change":"2024-04-25T01:43:17.424990+0000","last_active":"2024-04-25T01:44:46.918096+0000","last_peered":"2024-04-25T01:44:46.918096+0000","last_clean":"2024-04-25T01:44:46.918096+0000","last_became_active":"2024-04-25T01:43:17.424598+0000","last_became_peered":"2024-04-25T01:43:17.424598+0000","last_unstale":"2024-04-25T01:44:46.918096+0000","last_undegraded":"2024-04-25T01:44:46.918096+0000","last_fullsized":"2024-04-25T01:44:46.918096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:34.855089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.514597+0000","last_change":"2024-04-25T01:43:21.444138+0000","last_active":"2024-04-25T01:44:18.514597+0000","last_peered":"2024-04-25T01:44:18.514597+0000","last_clean":"2024-04-25T01:44:18.514597+0000","last_became_active":"2024-04-25T01:43:21.443283+0000","last_became_peered":"2024-04-25T01:43:21.443283+0000","last_unstale":"2024-04-25T01:44:18.514597+0000","last_undegraded":"2024-04-25T01:44:18.514597+0000","last_fullsized":"2024-04-25T01:44:18.514597+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:46.275789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.510471+0000","last_change":"2024-04-25T01:43:17.027892+0000","last_active":"2024-04-25T01:44:18.510471+0000","last_peered":"2024-04-25T01:44:18.510471+0000","last_clean":"2024-04-25T01:44:18.510471+0000","last_became_active":"2024-04-25T01:43:17.027476+0000","last_became_peered":"2024-04-25T01:43:17.027476+0000","last_unstale":"2024-04-25T01:44:18.510471+0000","last_undegraded":"2024-04-25T01:44:18.510471+0000","last_fullsized":"2024-04-25T01:44:18.510471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:31.079700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.924320+0000","last_change":"2024-04-25T01:43:17.431491+0000","last_active":"2024-04-25T01:44:46.924320+0000","last_peered":"2024-04-25T01:44:46.924320+0000","last_clean":"2024-04-25T01:44:46.924320+0000","last_became_active":"2024-04-25T01:43:17.431160+0000","last_became_peered":"2024-04-25T01:43:17.431160+0000","last_unstale":"2024-04-25T01:44:46.924320+0000","last_undegraded":"2024-04-25T01:44:46.924320+0000","last_fullsized":"2024-04-25T01:44:46.924320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:03.612857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509454+0000","last_change":"2024-04-25T01:43:21.626398+0000","last_active":"2024-04-25T01:44:18.509454+0000","last_peered":"2024-04-25T01:44:18.509454+0000","last_clean":"2024-04-25T01:44:18.509454+0000","last_became_active":"2024-04-25T01:43:21.625856+0000","last_became_peered":"2024-04-25T01:43:21.625856+0000","last_unstale":"2024-04-25T01:44:18.509454+0000","last_undegraded":"2024-04-25T01:44:18.509454+0000","last_fullsized":"2024-04-25T01:44:18.509454+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:53.000912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509470+0000","last_change":"2024-04-25T01:43:17.027564+0000","last_active":"2024-04-25T01:44:18.509470+0000","last_peered":"2024-04-25T01:44:18.509470+0000","last_clean":"2024-04-25T01:44:18.509470+0000","last_became_active":"2024-04-25T01:43:17.027218+0000","last_became_peered":"2024-04-25T01:43:17.027218+0000","last_unstale":"2024-04-25T01:44:18.509470+0000","last_undegraded":"2024-04-25T01:44:18.509470+0000","last_fullsized":"2024-04-25T01:44:18.509470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:52.658023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.914564+0000","last_change":"2024-04-25T01:43:17.431647+0000","last_active":"2024-04-25T01:44:46.914564+0000","last_peered":"2024-04-25T01:44:46.914564+0000","last_clean":"2024-04-25T01:44:46.914564+0000","last_became_active":"2024-04-25T01:43:17.431048+0000","last_became_peered":"2024-04-25T01:43:17.431048+0000","last_unstale":"2024-04-25T01:44:46.914564+0000","last_undegraded":"2024-04-25T01:44:46.914564+0000","last_fullsized":"2024-04-25T01:44:46.914564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:30.124585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512606+0000","last_change":"2024-04-25T01:43:21.450326+0000","last_active":"2024-04-25T01:44:18.512606+0000","last_peered":"2024-04-25T01:44:18.512606+0000","last_clean":"2024-04-25T01:44:18.512606+0000","last_became_active":"2024-04-25T01:43:21.449925+0000","last_became_peered":"2024-04-25T01:43:21.449925+0000","last_unstale":"2024-04-25T01:44:18.512606+0000","last_undegraded":"2024-04-25T01:44:18.512606+0000","last_fullsized":"2024-04-25T01:44:18.512606+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:39.896905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":102,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:51.532717+0000","last_change":"2024-04-25T01:43:17.032223+0000","last_active":"2024-04-25T01:44:51.532717+0000","last_peered":"2024-04-25T01:44:51.532717+0000","last_clean":"2024-04-25T01:44:51.532717+0000","last_became_active":"2024-04-25T01:43:17.031774+0000","last_became_peered":"2024-04-25T01:43:17.031774+0000","last_unstale":"2024-04-25T01:44:51.532717+0000","last_undegraded":"2024-04-25T01:44:51.532717+0000","last_fullsized":"2024-04-25T01:44:51.532717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:03.730850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.922359+0000","last_change":"2024-04-25T01:43:17.425756+0000","last_active":"2024-04-25T01:44:46.922359+0000","last_peered":"2024-04-25T01:44:46.922359+0000","last_clean":"2024-04-25T01:44:46.922359+0000","last_became_active":"2024-04-25T01:43:17.425433+0000","last_became_peered":"2024-04-25T01:43:17.425433+0000","last_unstale":"2024-04-25T01:44:46.922359+0000","last_undegraded":"2024-04-25T01:44:46.922359+0000","last_fullsized":"2024-04-25T01:44:46.922359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:29.776771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511889+0000","last_change":"2024-04-25T01:43:21.625898+0000","last_active":"2024-04-25T01:44:18.511889+0000","last_peered":"2024-04-25T01:44:18.511889+0000","last_clean":"2024-04-25T01:44:18.511889+0000","last_became_active":"2024-04-25T01:43:21.625460+0000","last_became_peered":"2024-04-25T01:43:21.625460+0000","last_unstale":"2024-04-25T01:44:18.511889+0000","last_undegraded":"2024-04-25T01:44:18.511889+0000","last_fullsized":"2024-04-25T01:44:18.511889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:31.199172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.906587+0000","last_change":"2024-04-25T01:44:18.515838+0000","last_active":"2024-04-25T01:44:46.906587+0000","last_peered":"2024-04-25T01:44:46.906587+0000","last_clean":"2024-04-25T01:44:46.906587+0000","last_became_active":"2024-04-25T01:44:18.515440+0000","last_became_peered":"2024-04-25T01:44:18.515440+0000","last_unstale":"2024-04-25T01:44:46.906587+0000","last_undegraded":"2024-04-25T01:44:46.906587+0000","last_fullsized":"2024-04-25T01:44:46.906587+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:54:33.072084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:52.531524+0000","last_change":"2024-04-25T01:43:17.028904+0000","last_active":"2024-04-25T01:44:52.531524+0000","last_peered":"2024-04-25T01:44:52.531524+0000","last_clean":"2024-04-25T01:44:52.531524+0000","last_became_active":"2024-04-25T01:43:17.028608+0000","last_became_peered":"2024-04-25T01:43:17.028608+0000","last_unstale":"2024-04-25T01:44:52.531524+0000","last_undegraded":"2024-04-25T01:44:52.531524+0000","last_fullsized":"2024-04-25T01:44:52.531524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:19.297926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.521409+0000","last_change":"2024-04-25T01:44:18.521409+0000","last_active":"2024-04-25T01:44:18.521409+0000","last_peered":"2024-04-25T01:44:18.521409+0000","last_clean":"2024-04-25T01:44:18.521409+0000","last_became_active":"2024-04-25T01:44:18.515062+0000","last_became_peered":"2024-04-25T01:44:18.515062+0000","last_unstale":"2024-04-25T01:44:18.521409+0000","last_undegraded":"2024-04-25T01:44:18.521409+0000","last_fullsized":"2024-04-25T01:44:18.521409+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:30.072124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919567+0000","last_change":"2024-04-25T01:43:17.428636+0000","last_active":"2024-04-25T01:44:46.919567+0000","last_peered":"2024-04-25T01:44:46.919567+0000","last_clean":"2024-04-25T01:44:46.919567+0000","last_became_active":"2024-04-25T01:43:17.428316+0000","last_became_peered":"2024-04-25T01:43:17.428316+0000","last_unstale":"2024-04-25T01:44:46.919567+0000","last_undegraded":"2024-04-25T01:44:46.919567+0000","last_fullsized":"2024-04-25T01:44:46.919567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:04.683846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512267+0000","last_change":"2024-04-25T01:43:17.020632+0000","last_active":"2024-04-25T01:44:18.512267+0000","last_peered":"2024-04-25T01:44:18.512267+0000","last_clean":"2024-04-25T01:44:18.512267+0000","last_became_active":"2024-04-25T01:43:17.020278+0000","last_became_peered":"2024-04-25T01:43:17.020278+0000","last_unstale":"2024-04-25T01:44:18.512267+0000","last_undegraded":"2024-04-25T01:44:18.512267+0000","last_fullsized":"2024-04-25T01:44:18.512267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:13.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512749+0000","last_change":"2024-04-25T01:42:05.003964+0000","last_active":"2024-04-25T01:44:18.512749+0000","last_peered":"2024-04-25T01:44:18.512749+0000","last_clean":"2024-04-25T01:44:18.512749+0000","last_became_active":"2024-04-25T01:42:05.003509+0000","last_became_peered":"2024-04-25T01:42:05.003509+0000","last_unstale":"2024-04-25T01:44:18.512749+0000","last_undegraded":"2024-04-25T01:44:18.512749+0000","last_fullsized":"2024-04-25T01:44:18.512749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:38:39.137002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:38:39.137002+0000","last_clean_scrub_stamp":"2024-04-25T01:38:39.137002+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:53.878838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.515627+0000","last_change":"2024-04-25T01:43:21.626345+0000","last_active":"2024-04-25T01:44:18.515627+0000","last_peered":"2024-04-25T01:44:18.515627+0000","last_clean":"2024-04-25T01:44:18.515627+0000","last_became_active":"2024-04-25T01:43:21.625884+0000","last_became_peered":"2024-04-25T01:43:21.625884+0000","last_unstale":"2024-04-25T01:44:18.515627+0000","last_undegraded":"2024-04-25T01:44:18.515627+0000","last_fullsized":"2024-04-25T01:44:18.515627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:37.961617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.916916+0000","last_change":"2024-04-25T01:43:17.421186+0000","last_active":"2024-04-25T01:44:46.916916+0000","last_peered":"2024-04-25T01:44:46.916916+0000","last_clean":"2024-04-25T01:44:46.916916+0000","last_became_active":"2024-04-25T01:43:17.420824+0000","last_became_peered":"2024-04-25T01:43:17.420824+0000","last_unstale":"2024-04-25T01:44:46.916916+0000","last_undegraded":"2024-04-25T01:44:46.916916+0000","last_fullsized":"2024-04-25T01:44:46.916916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:27.829126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512196+0000","last_change":"2024-04-25T01:43:17.034050+0000","last_active":"2024-04-25T01:44:18.512196+0000","last_peered":"2024-04-25T01:44:18.512196+0000","last_clean":"2024-04-25T01:44:18.512196+0000","last_became_active":"2024-04-25T01:43:17.033725+0000","last_became_peered":"2024-04-25T01:43:17.033725+0000","last_unstale":"2024-04-25T01:44:18.512196+0000","last_undegraded":"2024-04-25T01:44:18.512196+0000","last_fullsized":"2024-04-25T01:44:18.512196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:06.419265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511329+0000","last_change":"2024-04-25T01:43:21.443299+0000","last_active":"2024-04-25T01:44:18.511329+0000","last_peered":"2024-04-25T01:44:18.511329+0000","last_clean":"2024-04-25T01:44:18.511329+0000","last_became_active":"2024-04-25T01:43:21.442899+0000","last_became_peered":"2024-04-25T01:43:21.442899+0000","last_unstale":"2024-04-25T01:44:18.511329+0000","last_undegraded":"2024-04-25T01:44:18.511329+0000","last_fullsized":"2024-04-25T01:44:18.511329+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:51.720208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509960+0000","last_change":"2024-04-25T01:43:17.035136+0000","last_active":"2024-04-25T01:44:18.509960+0000","last_peered":"2024-04-25T01:44:18.509960+0000","last_clean":"2024-04-25T01:44:18.509960+0000","last_became_active":"2024-04-25T01:43:17.034289+0000","last_became_peered":"2024-04-25T01:43:17.034289+0000","last_unstale":"2024-04-25T01:44:18.509960+0000","last_undegraded":"2024-04-25T01:44:18.509960+0000","last_fullsized":"2024-04-25T01:44:18.509960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:07.450025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.908477+0000","last_change":"2024-04-25T01:43:17.414478+0000","last_active":"2024-04-25T01:44:46.908477+0000","last_peered":"2024-04-25T01:44:46.908477+0000","last_clean":"2024-04-25T01:44:46.908477+0000","last_became_active":"2024-04-25T01:43:17.414061+0000","last_became_peered":"2024-04-25T01:43:17.414061+0000","last_unstale":"2024-04-25T01:44:46.908477+0000","last_undegraded":"2024-04-25T01:44:46.908477+0000","last_fullsized":"2024-04-25T01:44:46.908477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:16.051518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.581417+0000","last_change":"2024-04-25T01:43:21.625963+0000","last_active":"2024-04-25T01:44:18.581417+0000","last_peered":"2024-04-25T01:44:18.581417+0000","last_clean":"2024-04-25T01:44:18.581417+0000","last_became_active":"2024-04-25T01:43:21.625563+0000","last_became_peered":"2024-04-25T01:43:21.625563+0000","last_unstale":"2024-04-25T01:44:18.581417+0000","last_undegraded":"2024-04-25T01:44:18.581417+0000","last_fullsized":"2024-04-25T01:44:18.581417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:11.217504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509914+0000","last_change":"2024-04-25T01:43:17.024391+0000","last_active":"2024-04-25T01:44:18.509914+0000","last_peered":"2024-04-25T01:44:18.509914+0000","last_clean":"2024-04-25T01:44:18.509914+0000","last_became_active":"2024-04-25T01:43:17.023994+0000","last_became_peered":"2024-04-25T01:43:17.023994+0000","last_unstale":"2024-04-25T01:44:18.509914+0000","last_undegraded":"2024-04-25T01:44:18.509914+0000","last_fullsized":"2024-04-25T01:44:18.509914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:56.489383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.914837+0000","last_change":"2024-04-25T01:43:17.435357+0000","last_active":"2024-04-25T01:44:46.914837+0000","last_peered":"2024-04-25T01:44:46.914837+0000","last_clean":"2024-04-25T01:44:46.914837+0000","last_became_active":"2024-04-25T01:43:17.435090+0000","last_became_peered":"2024-04-25T01:43:17.435090+0000","last_unstale":"2024-04-25T01:44:46.914837+0000","last_undegraded":"2024-04-25T01:44:46.914837+0000","last_fullsized":"2024-04-25T01:44:46.914837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:49.181850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.515185+0000","last_change":"2024-04-25T01:43:21.441314+0000","last_active":"2024-04-25T01:44:18.515185+0000","last_peered":"2024-04-25T01:44:18.515185+0000","last_clean":"2024-04-25T01:44:18.515185+0000","last_became_active":"2024-04-25T01:43:21.440896+0000","last_became_peered":"2024-04-25T01:43:21.440896+0000","last_unstale":"2024-04-25T01:44:18.515185+0000","last_undegraded":"2024-04-25T01:44:18.515185+0000","last_fullsized":"2024-04-25T01:44:18.515185+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:36.091128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512284+0000","last_change":"2024-04-25T01:43:17.031525+0000","last_active":"2024-04-25T01:44:18.512284+0000","last_peered":"2024-04-25T01:44:18.512284+0000","last_clean":"2024-04-25T01:44:18.512284+0000","last_became_active":"2024-04-25T01:43:17.031203+0000","last_became_peered":"2024-04-25T01:43:17.031203+0000","last_unstale":"2024-04-25T01:44:18.512284+0000","last_undegraded":"2024-04-25T01:44:18.512284+0000","last_fullsized":"2024-04-25T01:44:18.512284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:22.929179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919567+0000","last_change":"2024-04-25T01:43:17.425981+0000","last_active":"2024-04-25T01:44:46.919567+0000","last_peered":"2024-04-25T01:44:46.919567+0000","last_clean":"2024-04-25T01:44:46.919567+0000","last_became_active":"2024-04-25T01:43:17.425674+0000","last_became_peered":"2024-04-25T01:43:17.425674+0000","last_unstale":"2024-04-25T01:44:46.919567+0000","last_undegraded":"2024-04-25T01:44:46.919567+0000","last_fullsized":"2024-04-25T01:44:46.919567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:58.441774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511438+0000","last_change":"2024-04-25T01:43:21.626389+0000","last_active":"2024-04-25T01:44:18.511438+0000","last_peered":"2024-04-25T01:44:18.511438+0000","last_clean":"2024-04-25T01:44:18.511438+0000","last_became_active":"2024-04-25T01:43:21.625859+0000","last_became_peered":"2024-04-25T01:43:21.625859+0000","last_unstale":"2024-04-25T01:44:18.511438+0000","last_undegraded":"2024-04-25T01:44:18.511438+0000","last_fullsized":"2024-04-25T01:44:18.511438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:51.372126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.730959+0000","last_change":"2024-04-25T01:43:17.022903+0000","last_active":"2024-04-25T01:44:18.730959+0000","last_peered":"2024-04-25T01:44:18.730959+0000","last_clean":"2024-04-25T01:44:18.730959+0000","last_became_active":"2024-04-25T01:43:17.022564+0000","last_became_peered":"2024-04-25T01:43:17.022564+0000","last_unstale":"2024-04-25T01:44:18.730959+0000","last_undegraded":"2024-04-25T01:44:18.730959+0000","last_fullsized":"2024-04-25T01:44:18.730959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:01.277500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.906202+0000","last_change":"2024-04-25T01:43:17.422855+0000","last_active":"2024-04-25T01:44:46.906202+0000","last_peered":"2024-04-25T01:44:46.906202+0000","last_clean":"2024-04-25T01:44:46.906202+0000","last_became_active":"2024-04-25T01:43:17.422490+0000","last_became_peered":"2024-04-25T01:43:17.422490+0000","last_unstale":"2024-04-25T01:44:46.906202+0000","last_undegraded":"2024-04-25T01:44:46.906202+0000","last_fullsized":"2024-04-25T01:44:46.906202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:40.016044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509793+0000","last_change":"2024-04-25T01:43:21.440080+0000","last_active":"2024-04-25T01:44:18.509793+0000","last_peered":"2024-04-25T01:44:18.509793+0000","last_clean":"2024-04-25T01:44:18.509793+0000","last_became_active":"2024-04-25T01:43:21.439669+0000","last_became_peered":"2024-04-25T01:43:21.439669+0000","last_unstale":"2024-04-25T01:44:18.509793+0000","last_undegraded":"2024-04-25T01:44:18.509793+0000","last_fullsized":"2024-04-25T01:44:18.509793+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:34.143210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.581111+0000","last_change":"2024-04-25T01:43:17.035521+0000","last_active":"2024-04-25T01:44:18.581111+0000","last_peered":"2024-04-25T01:44:18.581111+0000","last_clean":"2024-04-25T01:44:18.581111+0000","last_became_active":"2024-04-25T01:43:17.033762+0000","last_became_peered":"2024-04-25T01:43:17.033762+0000","last_unstale":"2024-04-25T01:44:18.581111+0000","last_undegraded":"2024-04-25T01:44:18.581111+0000","last_fullsized":"2024-04-25T01:44:18.581111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:04.762578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.922112+0000","last_change":"2024-04-25T01:43:17.426952+0000","last_active":"2024-04-25T01:44:46.922112+0000","last_peered":"2024-04-25T01:44:46.922112+0000","last_clean":"2024-04-25T01:44:46.922112+0000","last_became_active":"2024-04-25T01:43:17.426573+0000","last_became_peered":"2024-04-25T01:43:17.426573+0000","last_unstale":"2024-04-25T01:44:46.922112+0000","last_undegraded":"2024-04-25T01:44:46.922112+0000","last_fullsized":"2024-04-25T01:44:46.922112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:33.807254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511797+0000","last_change":"2024-04-25T01:43:21.453295+0000","last_active":"2024-04-25T01:44:18.511797+0000","last_peered":"2024-04-25T01:44:18.511797+0000","last_clean":"2024-04-25T01:44:18.511797+0000","last_became_active":"2024-04-25T01:43:21.452031+0000","last_became_peered":"2024-04-25T01:43:21.452031+0000","last_unstale":"2024-04-25T01:44:18.511797+0000","last_undegraded":"2024-04-25T01:44:18.511797+0000","last_fullsized":"2024-04-25T01:44:18.511797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:23.956638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.515227+0000","last_change":"2024-04-25T01:43:17.029253+0000","last_active":"2024-04-25T01:44:18.515227+0000","last_peered":"2024-04-25T01:44:18.515227+0000","last_clean":"2024-04-25T01:44:18.515227+0000","last_became_active":"2024-04-25T01:43:17.028949+0000","last_became_peered":"2024-04-25T01:43:17.028949+0000","last_unstale":"2024-04-25T01:44:18.515227+0000","last_undegraded":"2024-04-25T01:44:18.515227+0000","last_fullsized":"2024-04-25T01:44:18.515227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:05.101916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.920902+0000","last_change":"2024-04-25T01:43:17.431319+0000","last_active":"2024-04-25T01:44:46.920902+0000","last_peered":"2024-04-25T01:44:46.920902+0000","last_clean":"2024-04-25T01:44:46.920902+0000","last_became_active":"2024-04-25T01:43:17.431007+0000","last_became_peered":"2024-04-25T01:43:17.431007+0000","last_unstale":"2024-04-25T01:44:46.920902+0000","last_undegraded":"2024-04-25T01:44:46.920902+0000","last_fullsized":"2024-04-25T01:44:46.920902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:38.999286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511443+0000","last_change":"2024-04-25T01:43:21.625931+0000","last_active":"2024-04-25T01:44:18.511443+0000","last_peered":"2024-04-25T01:44:18.511443+0000","last_clean":"2024-04-25T01:44:18.511443+0000","last_became_active":"2024-04-25T01:43:21.625460+0000","last_became_peered":"2024-04-25T01:43:21.625460+0000","last_unstale":"2024-04-25T01:44:18.511443+0000","last_undegraded":"2024-04-25T01:44:18.511443+0000","last_fullsized":"2024-04-25T01:44:18.511443+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:59.665909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509835+0000","last_change":"2024-04-25T01:43:17.033937+0000","last_active":"2024-04-25T01:44:18.509835+0000","last_peered":"2024-04-25T01:44:18.509835+0000","last_clean":"2024-04-25T01:44:18.509835+0000","last_became_active":"2024-04-25T01:43:17.033629+0000","last_became_peered":"2024-04-25T01:43:17.033629+0000","last_unstale":"2024-04-25T01:44:18.509835+0000","last_undegraded":"2024-04-25T01:44:18.509835+0000","last_fullsized":"2024-04-25T01:44:18.509835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:05.531624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.936394+0000","last_change":"2024-04-25T01:43:17.436068+0000","last_active":"2024-04-25T01:44:46.936394+0000","last_peered":"2024-04-25T01:44:46.936394+0000","last_clean":"2024-04-25T01:44:46.936394+0000","last_became_active":"2024-04-25T01:43:17.435383+0000","last_became_peered":"2024-04-25T01:43:17.435383+0000","last_unstale":"2024-04-25T01:44:46.936394+0000","last_undegraded":"2024-04-25T01:44:46.936394+0000","last_fullsized":"2024-04-25T01:44:46.936394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:00.087923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.730978+0000","last_change":"2024-04-25T01:43:21.441696+0000","last_active":"2024-04-25T01:44:18.730978+0000","last_peered":"2024-04-25T01:44:18.730978+0000","last_clean":"2024-04-25T01:44:18.730978+0000","last_became_active":"2024-04-25T01:43:21.441263+0000","last_became_peered":"2024-04-25T01:43:21.441263+0000","last_unstale":"2024-04-25T01:44:18.730978+0000","last_undegraded":"2024-04-25T01:44:18.730978+0000","last_fullsized":"2024-04-25T01:44:18.730978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:57.263465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":107,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:52.550772+0000","last_change":"2024-04-25T01:43:17.033841+0000","last_active":"2024-04-25T01:44:52.550772+0000","last_peered":"2024-04-25T01:44:52.550772+0000","last_clean":"2024-04-25T01:44:52.550772+0000","last_became_active":"2024-04-25T01:43:17.033477+0000","last_became_peered":"2024-04-25T01:43:17.033477+0000","last_unstale":"2024-04-25T01:44:52.550772+0000","last_undegraded":"2024-04-25T01:44:52.550772+0000","last_fullsized":"2024-04-25T01:44:52.550772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:43.717525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.936319+0000","last_change":"2024-04-25T01:43:17.430744+0000","last_active":"2024-04-25T01:44:46.936319+0000","last_peered":"2024-04-25T01:44:46.936319+0000","last_clean":"2024-04-25T01:44:46.936319+0000","last_became_active":"2024-04-25T01:43:17.430434+0000","last_became_peered":"2024-04-25T01:43:17.430434+0000","last_unstale":"2024-04-25T01:44:46.936319+0000","last_undegraded":"2024-04-25T01:44:46.936319+0000","last_fullsized":"2024-04-25T01:44:46.936319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:17.895901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511855+0000","last_change":"2024-04-25T01:43:21.451604+0000","last_active":"2024-04-25T01:44:18.511855+0000","last_peered":"2024-04-25T01:44:18.511855+0000","last_clean":"2024-04-25T01:44:18.511855+0000","last_became_active":"2024-04-25T01:43:21.451178+0000","last_became_peered":"2024-04-25T01:43:21.451178+0000","last_unstale":"2024-04-25T01:44:18.511855+0000","last_undegraded":"2024-04-25T01:44:18.511855+0000","last_fullsized":"2024-04-25T01:44:18.511855+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:17.387498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.515741+0000","last_change":"2024-04-25T01:43:17.025580+0000","last_active":"2024-04-25T01:44:18.515741+0000","last_peered":"2024-04-25T01:44:18.515741+0000","last_clean":"2024-04-25T01:44:18.515741+0000","last_became_active":"2024-04-25T01:43:17.025267+0000","last_became_peered":"2024-04-25T01:43:17.025267+0000","last_unstale":"2024-04-25T01:44:18.515741+0000","last_undegraded":"2024-04-25T01:44:18.515741+0000","last_fullsized":"2024-04-25T01:44:18.515741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:04.922572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.924107+0000","last_change":"2024-04-25T01:44:18.517600+0000","last_active":"2024-04-25T01:44:46.924107+0000","last_peered":"2024-04-25T01:44:46.924107+0000","last_clean":"2024-04-25T01:44:46.924107+0000","last_became_active":"2024-04-25T01:44:18.517169+0000","last_became_peered":"2024-04-25T01:44:18.517169+0000","last_unstale":"2024-04-25T01:44:46.924107+0000","last_undegraded":"2024-04-25T01:44:46.924107+0000","last_fullsized":"2024-04-25T01:44:46.924107+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:11.389592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.513792+0000","last_change":"2024-04-25T01:43:21.437988+0000","last_active":"2024-04-25T01:44:18.513792+0000","last_peered":"2024-04-25T01:44:18.513792+0000","last_clean":"2024-04-25T01:44:18.513792+0000","last_became_active":"2024-04-25T01:43:21.437523+0000","last_became_peered":"2024-04-25T01:43:21.437523+0000","last_unstale":"2024-04-25T01:44:18.513792+0000","last_undegraded":"2024-04-25T01:44:18.513792+0000","last_fullsized":"2024-04-25T01:44:18.513792+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:09.076123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512093+0000","last_change":"2024-04-25T01:43:17.023058+0000","last_active":"2024-04-25T01:44:18.512093+0000","last_peered":"2024-04-25T01:44:18.512093+0000","last_clean":"2024-04-25T01:44:18.512093+0000","last_became_active":"2024-04-25T01:43:17.022730+0000","last_became_peered":"2024-04-25T01:43:17.022730+0000","last_unstale":"2024-04-25T01:44:18.512093+0000","last_undegraded":"2024-04-25T01:44:18.512093+0000","last_fullsized":"2024-04-25T01:44:18.512093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:12.279161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.929742+0000","last_change":"2024-04-25T01:43:17.428832+0000","last_active":"2024-04-25T01:44:46.929742+0000","last_peered":"2024-04-25T01:44:46.929742+0000","last_clean":"2024-04-25T01:44:46.929742+0000","last_became_active":"2024-04-25T01:43:17.428493+0000","last_became_peered":"2024-04-25T01:43:17.428493+0000","last_unstale":"2024-04-25T01:44:46.929742+0000","last_undegraded":"2024-04-25T01:44:46.929742+0000","last_fullsized":"2024-04-25T01:44:46.929742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:47.224987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.923282+0000","last_change":"2024-04-25T01:43:17.429227+0000","last_active":"2024-04-25T01:44:46.923282+0000","last_peered":"2024-04-25T01:44:46.923282+0000","last_clean":"2024-04-25T01:44:46.923282+0000","last_became_active":"2024-04-25T01:43:17.428693+0000","last_became_peered":"2024-04-25T01:43:17.428693+0000","last_unstale":"2024-04-25T01:44:46.923282+0000","last_undegraded":"2024-04-25T01:44:46.923282+0000","last_fullsized":"2024-04-25T01:44:46.923282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:58.511414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.917515+0000","last_change":"2024-04-25T01:44:18.518461+0000","last_active":"2024-04-25T01:44:46.917515+0000","last_peered":"2024-04-25T01:44:46.917515+0000","last_clean":"2024-04-25T01:44:46.917515+0000","last_became_active":"2024-04-25T01:44:18.518110+0000","last_became_peered":"2024-04-25T01:44:18.518110+0000","last_unstale":"2024-04-25T01:44:46.917515+0000","last_undegraded":"2024-04-25T01:44:46.917515+0000","last_fullsized":"2024-04-25T01:44:46.917515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:58.551833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.920042+0000","last_change":"2024-04-25T01:43:17.421214+0000","last_active":"2024-04-25T01:44:46.920042+0000","last_peered":"2024-04-25T01:44:46.920042+0000","last_clean":"2024-04-25T01:44:46.920042+0000","last_became_active":"2024-04-25T01:43:17.420872+0000","last_became_peered":"2024-04-25T01:43:17.420872+0000","last_unstale":"2024-04-25T01:44:46.920042+0000","last_undegraded":"2024-04-25T01:44:46.920042+0000","last_fullsized":"2024-04-25T01:44:46.920042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:48.066191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.918399+0000","last_change":"2024-04-25T01:43:17.426011+0000","last_active":"2024-04-25T01:44:46.918399+0000","last_peered":"2024-04-25T01:44:46.918399+0000","last_clean":"2024-04-25T01:44:46.918399+0000","last_became_active":"2024-04-25T01:43:17.425629+0000","last_became_peered":"2024-04-25T01:43:17.425629+0000","last_unstale":"2024-04-25T01:44:46.918399+0000","last_undegraded":"2024-04-25T01:44:46.918399+0000","last_fullsized":"2024-04-25T01:44:46.918399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:12.606749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919975+0000","last_change":"2024-04-25T01:43:17.421374+0000","last_active":"2024-04-25T01:44:46.919975+0000","last_peered":"2024-04-25T01:44:46.919975+0000","last_clean":"2024-04-25T01:44:46.919975+0000","last_became_active":"2024-04-25T01:43:17.420864+0000","last_became_peered":"2024-04-25T01:43:17.420864+0000","last_unstale":"2024-04-25T01:44:46.919975+0000","last_undegraded":"2024-04-25T01:44:46.919975+0000","last_fullsized":"2024-04-25T01:44:46.919975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:17.517927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.922793+0000","last_change":"2024-04-25T01:43:17.431131+0000","last_active":"2024-04-25T01:44:46.922793+0000","last_peered":"2024-04-25T01:44:46.922793+0000","last_clean":"2024-04-25T01:44:46.922793+0000","last_became_active":"2024-04-25T01:43:17.430809+0000","last_became_peered":"2024-04-25T01:43:17.430809+0000","last_unstale":"2024-04-25T01:44:46.922793+0000","last_undegraded":"2024-04-25T01:44:46.922793+0000","last_fullsized":"2024-04-25T01:44:46.922793+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:48.261501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.930562+0000","last_change":"2024-04-25T01:43:17.419104+0000","last_active":"2024-04-25T01:44:46.930562+0000","last_peered":"2024-04-25T01:44:46.930562+0000","last_clean":"2024-04-25T01:44:46.930562+0000","last_became_active":"2024-04-25T01:43:17.418799+0000","last_became_peered":"2024-04-25T01:43:17.418799+0000","last_unstale":"2024-04-25T01:44:46.930562+0000","last_undegraded":"2024-04-25T01:44:46.930562+0000","last_fullsized":"2024-04-25T01:44:46.930562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:46.000847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.917714+0000","last_change":"2024-04-25T01:43:17.425801+0000","last_active":"2024-04-25T01:44:46.917714+0000","last_peered":"2024-04-25T01:44:46.917714+0000","last_clean":"2024-04-25T01:44:46.917714+0000","last_became_active":"2024-04-25T01:43:17.425133+0000","last_became_peered":"2024-04-25T01:43:17.425133+0000","last_unstale":"2024-04-25T01:44:46.917714+0000","last_undegraded":"2024-04-25T01:44:46.917714+0000","last_fullsized":"2024-04-25T01:44:46.917714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:54.229265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919025+0000","last_change":"2024-04-25T01:43:17.423931+0000","last_active":"2024-04-25T01:44:46.919025+0000","last_peered":"2024-04-25T01:44:46.919025+0000","last_clean":"2024-04-25T01:44:46.919025+0000","last_became_active":"2024-04-25T01:43:17.423477+0000","last_became_peered":"2024-04-25T01:43:17.423477+0000","last_unstale":"2024-04-25T01:44:46.919025+0000","last_undegraded":"2024-04-25T01:44:46.919025+0000","last_fullsized":"2024-04-25T01:44:46.919025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:31.538936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919483+0000","last_change":"2024-04-25T01:43:17.428469+0000","last_active":"2024-04-25T01:44:46.919483+0000","last_peered":"2024-04-25T01:44:46.919483+0000","last_clean":"2024-04-25T01:44:46.919483+0000","last_became_active":"2024-04-25T01:43:17.428121+0000","last_became_peered":"2024-04-25T01:43:17.428121+0000","last_unstale":"2024-04-25T01:44:46.919483+0000","last_undegraded":"2024-04-25T01:44:46.919483+0000","last_fullsized":"2024-04-25T01:44:46.919483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:27.563439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.937792+0000","last_change":"2024-04-25T01:43:17.433014+0000","last_active":"2024-04-25T01:44:46.937792+0000","last_peered":"2024-04-25T01:44:46.937792+0000","last_clean":"2024-04-25T01:44:46.937792+0000","last_became_active":"2024-04-25T01:43:17.432568+0000","last_became_peered":"2024-04-25T01:43:17.432568+0000","last_unstale":"2024-04-25T01:44:46.937792+0000","last_undegraded":"2024-04-25T01:44:46.937792+0000","last_fullsized":"2024-04-25T01:44:46.937792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:18.494922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.920697+0000","last_change":"2024-04-25T01:43:17.436079+0000","last_active":"2024-04-25T01:44:46.920697+0000","last_peered":"2024-04-25T01:44:46.920697+0000","last_clean":"2024-04-25T01:44:46.920697+0000","last_became_active":"2024-04-25T01:43:17.435735+0000","last_became_peered":"2024-04-25T01:43:17.435735+0000","last_unstale":"2024-04-25T01:44:46.920697+0000","last_undegraded":"2024-04-25T01:44:46.920697+0000","last_fullsized":"2024-04-25T01:44:46.920697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:59.593710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.924572+0000","last_change":"2024-04-25T01:43:17.424956+0000","last_active":"2024-04-25T01:44:46.924572+0000","last_peered":"2024-04-25T01:44:46.924572+0000","last_clean":"2024-04-25T01:44:46.924572+0000","last_became_active":"2024-04-25T01:43:17.424629+0000","last_became_peered":"2024-04-25T01:43:17.424629+0000","last_unstale":"2024-04-25T01:44:46.924572+0000","last_undegraded":"2024-04-25T01:44:46.924572+0000","last_fullsized":"2024-04-25T01:44:46.924572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:58.006952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245924,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71623,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":745,"num_read_kb":450,"num_write":253,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":214869,"data_compressed":4824,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351009,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":506404,"data_compressed":20650,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612646,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":1160921,"data_compressed":58104,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776171,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":509898,"data_compressed":20650,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037808,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":506180,"data_compressed":20650,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299445,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":1103350,"data_compressed":56496,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":501306,"data_compressed":20650,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":565941,"data_compressed":22258,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":503294,"data_compressed":20650,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":567225,"data_compressed":22258,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":501014,"data_compressed":20650,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":505912,"data_compressed":20650,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":1094304,"data_compressed":56496,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65357,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66641,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65873,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:44:56.704 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T01:44:56.949 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:44:57.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:56 smithi078 ceph-mon[29469]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:44:57.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:56 smithi152 ceph-mon[32465]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:44:57.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:56 smithi196 ceph-mon[32978]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:44:58.237 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:57 smithi078 ceph-mon[29469]: pgmap v342: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-25T01:44:58.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:57 smithi152 ceph-mon[32465]: pgmap v342: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-25T01:44:58.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:57 smithi196 ceph-mon[32978]: pgmap v342: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-25T01:44:58.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.588+0000 7f35b79e4700 1 -- 172.21.15.78:0/497237681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b01082b0 msgr2=0x7f35b0108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:58.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.588+0000 7f35b79e4700 1 --2- 172.21.15.78:0/497237681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b01082b0 0x7f35b0108690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009990 tx=0x7f35ac009ca0 comp rx=0 tx=0).stop 2024-04-25T01:44:58.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.588+0000 7f35b79e4700 1 -- 172.21.15.78:0/497237681 shutdown_connections 2024-04-25T01:44:58.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.588+0000 7f35b79e4700 1 --2- 172.21.15.78:0/497237681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b006b320 0x7f35b010f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:58.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.588+0000 7f35b79e4700 1 --2- 172.21.15.78:0/497237681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b01082b0 0x7f35b0108690 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:58.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.588+0000 7f35b79e4700 1 --2- 172.21.15.78:0/497237681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b006a980 0x7f35b006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:58.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.588+0000 7f35b79e4700 1 -- 172.21.15.78:0/497237681 >> 172.21.15.78:0/497237681 conn(0x7f35b0078100 msgr2=0x7f35b0078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:58.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.588+0000 7f35b79e4700 1 -- 172.21.15.78:0/497237681 shutdown_connections 2024-04-25T01:44:58.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.588+0000 7f35b79e4700 1 -- 172.21.15.78:0/497237681 wait complete. 2024-04-25T01:44:58.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.588+0000 7f35b79e4700 1 Processor -- start 2024-04-25T01:44:58.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.588+0000 7f35b79e4700 1 -- start start 2024-04-25T01:44:58.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b79e4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b006a980 0x7f35b019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:58.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b79e4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b006b320 0x7f35b019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:58.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b79e4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b01082b0 0x7f35b0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:58.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b79e4700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35b0112570 con 0x7f35b006b320 2024-04-25T01:44:58.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b79e4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f35b01123f0 con 0x7f35b01082b0 2024-04-25T01:44:58.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b79e4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35b01126f0 con 0x7f35b006a980 2024-04-25T01:44:58.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b4f7f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b006b320 0x7f35b019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:58.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b4f7f700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b006b320 0x7f35b019ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37446/0 (socket says 172.21.15.78:37446) 2024-04-25T01:44:58.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b4f7f700 1 -- 172.21.15.78:0/4110783644 learned_addr learned my addr 172.21.15.78:0/4110783644 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:44:58.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b5f81700 1 --2- 172.21.15.78:0/4110783644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b01082b0 0x7f35b0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b4f7f700 1 -- 172.21.15.78:0/4110783644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b006a980 msgr2=0x7f35b019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b4f7f700 1 --2- 172.21.15.78:0/4110783644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b006a980 0x7f35b019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b4f7f700 1 -- 172.21.15.78:0/4110783644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b01082b0 msgr2=0x7f35b0198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b4f7f700 1 --2- 172.21.15.78:0/4110783644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b01082b0 0x7f35b0198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b4f7f700 1 -- 172.21.15.78:0/4110783644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b0199010 con 0x7f35b006b320 2024-04-25T01:44:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b5f81700 1 --2- 172.21.15.78:0/4110783644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b01082b0 0x7f35b0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:44:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b4f7f700 1 --2- 172.21.15.78:0/4110783644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b006b320 0x7f35b019ebf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f35a000c9b0 tx=0x7f35a000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35a67fc700 1 -- 172.21.15.78:0/4110783644 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0007b00 con 0x7f35b006b320 2024-04-25T01:44:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35a67fc700 1 -- 172.21.15.78:0/4110783644 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a0007ca0 con 0x7f35b006b320 2024-04-25T01:44:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b79e4700 1 -- 172.21.15.78:0/4110783644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b0199300 con 0x7f35b006b320 2024-04-25T01:44:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35a67fc700 1 -- 172.21.15.78:0/4110783644 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0018780 con 0x7f35b006b320 2024-04-25T01:44:58.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.589+0000 7f35b79e4700 1 -- 172.21.15.78:0/4110783644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b01a5510 con 0x7f35b006b320 2024-04-25T01:44:58.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.590+0000 7f35b79e4700 1 -- 172.21.15.78:0/4110783644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b0103960 con 0x7f35b006b320 2024-04-25T01:44:58.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.591+0000 7f35a67fc700 1 -- 172.21.15.78:0/4110783644 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f35a000f450 con 0x7f35b006b320 2024-04-25T01:44:58.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.592+0000 7f35a67fc700 1 --2- 172.21.15.78:0/4110783644 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f359c0a2c20 0x7f359c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:58.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.592+0000 7f35b5780700 1 --2- 172.21.15.78:0/4110783644 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f359c0a2c20 0x7f359c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:58.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.592+0000 7f35a67fc700 1 -- 172.21.15.78:0/4110783644 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f35a00cf6b0 con 0x7f35b006b320 2024-04-25T01:44:58.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.592+0000 7f35b5780700 1 --2- 172.21.15.78:0/4110783644 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f359c0a2c20 0x7f359c0a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00b5c0 tx=0x7f35ac005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:58.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.596+0000 7f35a67fc700 1 -- 172.21.15.78:0/4110783644 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f35a0014030 con 0x7f35b006b320 2024-04-25T01:44:58.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.788+0000 7f35b79e4700 1 -- 172.21.15.78:0/4110783644 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35b0062670 con 0x7f359c0a2c20 2024-04-25T01:44:58.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.797+0000 7f35a67fc700 1 -- 172.21.15.78:0/4110783644 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340809 (secure 0 0 0) 0x7f35ac079360 con 0x7f359c0a2c20 2024-04-25T01:44:58.794 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:44:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.800+0000 7f35b79e4700 1 -- 172.21.15.78:0/4110783644 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f359c0a2c20 msgr2=0x7f359c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.800+0000 7f35b79e4700 1 --2- 172.21.15.78:0/4110783644 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f359c0a2c20 0x7f359c0a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00b5c0 tx=0x7f35ac005fb0 comp rx=0 tx=0).stop 2024-04-25T01:44:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.800+0000 7f35b79e4700 1 -- 172.21.15.78:0/4110783644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b006b320 msgr2=0x7f35b019ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.800+0000 7f35b79e4700 1 --2- 172.21.15.78:0/4110783644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b006b320 0x7f35b019ebf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f35a000c9b0 tx=0x7f35a000cd70 comp rx=0 tx=0).stop 2024-04-25T01:44:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.801+0000 7f35b79e4700 1 -- 172.21.15.78:0/4110783644 shutdown_connections 2024-04-25T01:44:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.801+0000 7f35b79e4700 1 --2- 172.21.15.78:0/4110783644 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f359c0a2c20 0x7f359c0a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.801+0000 7f35b79e4700 1 --2- 172.21.15.78:0/4110783644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35b006a980 0x7f35b019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.801+0000 7f35b79e4700 1 --2- 172.21.15.78:0/4110783644 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b01082b0 0x7f35b0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.801+0000 7f35b79e4700 1 --2- 172.21.15.78:0/4110783644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35b006b320 0x7f35b019ebf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.801+0000 7f35b79e4700 1 -- 172.21.15.78:0/4110783644 >> 172.21.15.78:0/4110783644 conn(0x7f35b0078100 msgr2=0x7f35b00ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.801+0000 7f35b79e4700 1 -- 172.21.15.78:0/4110783644 shutdown_connections 2024-04-25T01:44:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:44:58.801+0000 7f35b79e4700 1 -- 172.21.15.78:0/4110783644 wait complete. 2024-04-25T01:44:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-04-25T01:44:59.506 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-04-25T01:44:56.919157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662225,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":876,"num_read_kb":564,"num_write":436,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":860,"ondisk_log_size":860,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361592,"kb_used_data":12440,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580232,"statfs":{"total":1151940427776,"available":1151570157568,"internally_reserved":0,"allocated":12738560,"data_stored":8025749,"data_compressed":360162,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":59,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003780"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.935647+0000","last_change":"2024-04-25T01:43:17.423128+0000","last_active":"2024-04-25T01:44:46.935647+0000","last_peered":"2024-04-25T01:44:46.935647+0000","last_clean":"2024-04-25T01:44:46.935647+0000","last_became_active":"2024-04-25T01:43:17.422356+0000","last_became_peered":"2024-04-25T01:43:17.422356+0000","last_unstale":"2024-04-25T01:44:46.935647+0000","last_undegraded":"2024-04-25T01:44:46.935647+0000","last_fullsized":"2024-04-25T01:44:46.935647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:02:05.524727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.910330+0000","last_change":"2024-04-25T01:43:17.422866+0000","last_active":"2024-04-25T01:44:46.910330+0000","last_peered":"2024-04-25T01:44:46.910330+0000","last_clean":"2024-04-25T01:44:46.910330+0000","last_became_active":"2024-04-25T01:43:17.422574+0000","last_became_peered":"2024-04-25T01:43:17.422574+0000","last_unstale":"2024-04-25T01:44:46.910330+0000","last_undegraded":"2024-04-25T01:44:46.910330+0000","last_fullsized":"2024-04-25T01:44:46.910330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.842361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919618+0000","last_change":"2024-04-25T01:43:17.419320+0000","last_active":"2024-04-25T01:44:46.919618+0000","last_peered":"2024-04-25T01:44:46.919618+0000","last_clean":"2024-04-25T01:44:46.919618+0000","last_became_active":"2024-04-25T01:43:17.418988+0000","last_became_peered":"2024-04-25T01:43:17.418988+0000","last_unstale":"2024-04-25T01:44:46.919618+0000","last_undegraded":"2024-04-25T01:44:46.919618+0000","last_fullsized":"2024-04-25T01:44:46.919618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:18.324727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.921155+0000","last_change":"2024-04-25T01:43:17.430061+0000","last_active":"2024-04-25T01:44:46.921155+0000","last_peered":"2024-04-25T01:44:46.921155+0000","last_clean":"2024-04-25T01:44:46.921155+0000","last_became_active":"2024-04-25T01:43:17.429736+0000","last_became_peered":"2024-04-25T01:43:17.429736+0000","last_unstale":"2024-04-25T01:44:46.921155+0000","last_undegraded":"2024-04-25T01:44:46.921155+0000","last_fullsized":"2024-04-25T01:44:46.921155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:57.784221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.915675+0000","last_change":"2024-04-25T01:43:17.422876+0000","last_active":"2024-04-25T01:44:46.915675+0000","last_peered":"2024-04-25T01:44:46.915675+0000","last_clean":"2024-04-25T01:44:46.915675+0000","last_became_active":"2024-04-25T01:43:17.422523+0000","last_became_peered":"2024-04-25T01:43:17.422523+0000","last_unstale":"2024-04-25T01:44:46.915675+0000","last_undegraded":"2024-04-25T01:44:46.915675+0000","last_fullsized":"2024-04-25T01:44:46.915675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:14.328021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919150+0000","last_change":"2024-04-25T01:43:17.424941+0000","last_active":"2024-04-25T01:44:46.919150+0000","last_peered":"2024-04-25T01:44:46.919150+0000","last_clean":"2024-04-25T01:44:46.919150+0000","last_became_active":"2024-04-25T01:43:17.424620+0000","last_became_peered":"2024-04-25T01:43:17.424620+0000","last_unstale":"2024-04-25T01:44:46.919150+0000","last_undegraded":"2024-04-25T01:44:46.919150+0000","last_fullsized":"2024-04-25T01:44:46.919150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:06.111590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.917177+0000","last_change":"2024-04-25T01:43:17.429188+0000","last_active":"2024-04-25T01:44:46.917177+0000","last_peered":"2024-04-25T01:44:46.917177+0000","last_clean":"2024-04-25T01:44:46.917177+0000","last_became_active":"2024-04-25T01:43:17.428837+0000","last_became_peered":"2024-04-25T01:43:17.428837+0000","last_unstale":"2024-04-25T01:44:46.917177+0000","last_undegraded":"2024-04-25T01:44:46.917177+0000","last_fullsized":"2024-04-25T01:44:46.917177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:08.923208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.918468+0000","last_change":"2024-04-25T01:43:17.431402+0000","last_active":"2024-04-25T01:44:46.918468+0000","last_peered":"2024-04-25T01:44:46.918468+0000","last_clean":"2024-04-25T01:44:46.918468+0000","last_became_active":"2024-04-25T01:43:17.431044+0000","last_became_peered":"2024-04-25T01:43:17.431044+0000","last_unstale":"2024-04-25T01:44:46.918468+0000","last_undegraded":"2024-04-25T01:44:46.918468+0000","last_fullsized":"2024-04-25T01:44:46.918468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:20.731589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.937188+0000","last_change":"2024-04-25T01:43:17.430029+0000","last_active":"2024-04-25T01:44:46.937188+0000","last_peered":"2024-04-25T01:44:46.937188+0000","last_clean":"2024-04-25T01:44:46.937188+0000","last_became_active":"2024-04-25T01:43:17.428588+0000","last_became_peered":"2024-04-25T01:43:17.428588+0000","last_unstale":"2024-04-25T01:44:46.937188+0000","last_undegraded":"2024-04-25T01:44:46.937188+0000","last_fullsized":"2024-04-25T01:44:46.937188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:58.520175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.920958+0000","last_change":"2024-04-25T01:43:17.430377+0000","last_active":"2024-04-25T01:44:46.920958+0000","last_peered":"2024-04-25T01:44:46.920958+0000","last_clean":"2024-04-25T01:44:46.920958+0000","last_became_active":"2024-04-25T01:43:17.426609+0000","last_became_peered":"2024-04-25T01:43:17.426609+0000","last_unstale":"2024-04-25T01:44:46.920958+0000","last_undegraded":"2024-04-25T01:44:46.920958+0000","last_fullsized":"2024-04-25T01:44:46.920958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:10.696040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.936196+0000","last_change":"2024-04-25T01:43:17.431561+0000","last_active":"2024-04-25T01:44:46.936196+0000","last_peered":"2024-04-25T01:44:46.936196+0000","last_clean":"2024-04-25T01:44:46.936196+0000","last_became_active":"2024-04-25T01:43:17.431229+0000","last_became_peered":"2024-04-25T01:43:17.431229+0000","last_unstale":"2024-04-25T01:44:46.936196+0000","last_undegraded":"2024-04-25T01:44:46.936196+0000","last_fullsized":"2024-04-25T01:44:46.936196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:03.686373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.923858+0000","last_change":"2024-04-25T01:43:17.428442+0000","last_active":"2024-04-25T01:44:46.923858+0000","last_peered":"2024-04-25T01:44:46.923858+0000","last_clean":"2024-04-25T01:44:46.923858+0000","last_became_active":"2024-04-25T01:43:17.428052+0000","last_became_peered":"2024-04-25T01:43:17.428052+0000","last_unstale":"2024-04-25T01:44:46.923858+0000","last_undegraded":"2024-04-25T01:44:46.923858+0000","last_fullsized":"2024-04-25T01:44:46.923858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:06.097229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.913370+0000","last_change":"2024-04-25T01:43:17.420520+0000","last_active":"2024-04-25T01:44:46.913370+0000","last_peered":"2024-04-25T01:44:46.913370+0000","last_clean":"2024-04-25T01:44:46.913370+0000","last_became_active":"2024-04-25T01:43:17.420171+0000","last_became_peered":"2024-04-25T01:43:17.420171+0000","last_unstale":"2024-04-25T01:44:46.913370+0000","last_undegraded":"2024-04-25T01:44:46.913370+0000","last_fullsized":"2024-04-25T01:44:46.913370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:43.699472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.920581+0000","last_change":"2024-04-25T01:43:17.422114+0000","last_active":"2024-04-25T01:44:46.920581+0000","last_peered":"2024-04-25T01:44:46.920581+0000","last_clean":"2024-04-25T01:44:46.920581+0000","last_became_active":"2024-04-25T01:43:17.421729+0000","last_became_peered":"2024-04-25T01:43:17.421729+0000","last_unstale":"2024-04-25T01:44:46.920581+0000","last_undegraded":"2024-04-25T01:44:46.920581+0000","last_fullsized":"2024-04-25T01:44:46.920581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:00.955123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919833+0000","last_change":"2024-04-25T01:43:17.427344+0000","last_active":"2024-04-25T01:44:46.919833+0000","last_peered":"2024-04-25T01:44:46.919833+0000","last_clean":"2024-04-25T01:44:46.919833+0000","last_became_active":"2024-04-25T01:43:17.427044+0000","last_became_peered":"2024-04-25T01:43:17.427044+0000","last_unstale":"2024-04-25T01:44:46.919833+0000","last_undegraded":"2024-04-25T01:44:46.919833+0000","last_fullsized":"2024-04-25T01:44:46.919833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:24.564654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.933157+0000","last_change":"2024-04-25T01:43:17.427604+0000","last_active":"2024-04-25T01:44:46.933157+0000","last_peered":"2024-04-25T01:44:46.933157+0000","last_clean":"2024-04-25T01:44:46.933157+0000","last_became_active":"2024-04-25T01:43:17.427286+0000","last_became_peered":"2024-04-25T01:43:17.427286+0000","last_unstale":"2024-04-25T01:44:46.933157+0000","last_undegraded":"2024-04-25T01:44:46.933157+0000","last_fullsized":"2024-04-25T01:44:46.933157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:25.584150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.938412+0000","last_change":"2024-04-25T01:43:17.423401+0000","last_active":"2024-04-25T01:44:46.938412+0000","last_peered":"2024-04-25T01:44:46.938412+0000","last_clean":"2024-04-25T01:44:46.938412+0000","last_became_active":"2024-04-25T01:43:17.422807+0000","last_became_peered":"2024-04-25T01:43:17.422807+0000","last_unstale":"2024-04-25T01:44:46.938412+0000","last_undegraded":"2024-04-25T01:44:46.938412+0000","last_fullsized":"2024-04-25T01:44:46.938412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:58.461242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.935691+0000","last_change":"2024-04-25T01:43:17.422054+0000","last_active":"2024-04-25T01:44:46.935691+0000","last_peered":"2024-04-25T01:44:46.935691+0000","last_clean":"2024-04-25T01:44:46.935691+0000","last_became_active":"2024-04-25T01:43:17.421746+0000","last_became_peered":"2024-04-25T01:43:17.421746+0000","last_unstale":"2024-04-25T01:44:46.935691+0000","last_undegraded":"2024-04-25T01:44:46.935691+0000","last_fullsized":"2024-04-25T01:44:46.935691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:28.405852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.916459+0000","last_change":"2024-04-25T01:43:17.429288+0000","last_active":"2024-04-25T01:44:46.916459+0000","last_peered":"2024-04-25T01:44:46.916459+0000","last_clean":"2024-04-25T01:44:46.916459+0000","last_became_active":"2024-04-25T01:43:17.427794+0000","last_became_peered":"2024-04-25T01:43:17.427794+0000","last_unstale":"2024-04-25T01:44:46.916459+0000","last_undegraded":"2024-04-25T01:44:46.916459+0000","last_fullsized":"2024-04-25T01:44:46.916459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:33.003361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.515428+0000","last_change":"2024-04-25T01:43:21.438427+0000","last_active":"2024-04-25T01:44:18.515428+0000","last_peered":"2024-04-25T01:44:18.515428+0000","last_clean":"2024-04-25T01:44:18.515428+0000","last_became_active":"2024-04-25T01:43:21.438018+0000","last_became_peered":"2024-04-25T01:43:21.438018+0000","last_unstale":"2024-04-25T01:44:18.515428+0000","last_undegraded":"2024-04-25T01:44:18.515428+0000","last_fullsized":"2024-04-25T01:44:18.515428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:46.509437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":114,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:55.536310+0000","last_change":"2024-04-25T01:43:17.031110+0000","last_active":"2024-04-25T01:44:55.536310+0000","last_peered":"2024-04-25T01:44:55.536310+0000","last_clean":"2024-04-25T01:44:55.536310+0000","last_became_active":"2024-04-25T01:43:17.030806+0000","last_became_peered":"2024-04-25T01:43:17.030806+0000","last_unstale":"2024-04-25T01:44:55.536310+0000","last_undegraded":"2024-04-25T01:44:55.536310+0000","last_fullsized":"2024-04-25T01:44:55.536310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:52:12.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.917790+0000","last_change":"2024-04-25T01:43:17.428847+0000","last_active":"2024-04-25T01:44:46.917790+0000","last_peered":"2024-04-25T01:44:46.917790+0000","last_clean":"2024-04-25T01:44:46.917790+0000","last_became_active":"2024-04-25T01:43:17.428553+0000","last_became_peered":"2024-04-25T01:43:17.428553+0000","last_unstale":"2024-04-25T01:44:46.917790+0000","last_undegraded":"2024-04-25T01:44:46.917790+0000","last_fullsized":"2024-04-25T01:44:46.917790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:13.973275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.582673+0000","last_change":"2024-04-25T01:44:18.582673+0000","last_active":"2024-04-25T01:44:18.582673+0000","last_peered":"2024-04-25T01:44:18.582673+0000","last_clean":"2024-04-25T01:44:18.582673+0000","last_became_active":"2024-04-25T01:44:18.582312+0000","last_became_peered":"2024-04-25T01:44:18.582312+0000","last_unstale":"2024-04-25T01:44:18.582673+0000","last_undegraded":"2024-04-25T01:44:18.582673+0000","last_fullsized":"2024-04-25T01:44:18.582673+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:32:22.909294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.581774+0000","last_change":"2024-04-25T01:43:17.032954+0000","last_active":"2024-04-25T01:44:18.581774+0000","last_peered":"2024-04-25T01:44:18.581774+0000","last_clean":"2024-04-25T01:44:18.581774+0000","last_became_active":"2024-04-25T01:43:17.032627+0000","last_became_peered":"2024-04-25T01:43:17.032627+0000","last_unstale":"2024-04-25T01:44:18.581774+0000","last_undegraded":"2024-04-25T01:44:18.581774+0000","last_fullsized":"2024-04-25T01:44:18.581774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:31:07.167227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.917301+0000","last_change":"2024-04-25T01:43:17.419451+0000","last_active":"2024-04-25T01:44:46.917301+0000","last_peered":"2024-04-25T01:44:46.917301+0000","last_clean":"2024-04-25T01:44:46.917301+0000","last_became_active":"2024-04-25T01:43:17.419111+0000","last_became_peered":"2024-04-25T01:43:17.419111+0000","last_unstale":"2024-04-25T01:44:46.917301+0000","last_undegraded":"2024-04-25T01:44:46.917301+0000","last_fullsized":"2024-04-25T01:44:46.917301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:41.284248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512163+0000","last_change":"2024-04-25T01:43:21.452525+0000","last_active":"2024-04-25T01:44:18.512163+0000","last_peered":"2024-04-25T01:44:18.512163+0000","last_clean":"2024-04-25T01:44:18.512163+0000","last_became_active":"2024-04-25T01:43:21.451834+0000","last_became_peered":"2024-04-25T01:43:21.451834+0000","last_unstale":"2024-04-25T01:44:18.512163+0000","last_undegraded":"2024-04-25T01:44:18.512163+0000","last_fullsized":"2024-04-25T01:44:18.512163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:34.866314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.510174+0000","last_change":"2024-04-25T01:43:17.035737+0000","last_active":"2024-04-25T01:44:18.510174+0000","last_peered":"2024-04-25T01:44:18.510174+0000","last_clean":"2024-04-25T01:44:18.510174+0000","last_became_active":"2024-04-25T01:43:17.034709+0000","last_became_peered":"2024-04-25T01:43:17.034709+0000","last_unstale":"2024-04-25T01:44:18.510174+0000","last_undegraded":"2024-04-25T01:44:18.510174+0000","last_fullsized":"2024-04-25T01:44:18.510174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:29:50.419193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.936816+0000","last_change":"2024-04-25T01:43:17.422530+0000","last_active":"2024-04-25T01:44:46.936816+0000","last_peered":"2024-04-25T01:44:46.936816+0000","last_clean":"2024-04-25T01:44:46.936816+0000","last_became_active":"2024-04-25T01:43:17.422197+0000","last_became_peered":"2024-04-25T01:43:17.422197+0000","last_unstale":"2024-04-25T01:44:46.936816+0000","last_undegraded":"2024-04-25T01:44:46.936816+0000","last_fullsized":"2024-04-25T01:44:46.936816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:25.250947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511486+0000","last_change":"2024-04-25T01:43:21.446869+0000","last_active":"2024-04-25T01:44:18.511486+0000","last_peered":"2024-04-25T01:44:18.511486+0000","last_clean":"2024-04-25T01:44:18.511486+0000","last_became_active":"2024-04-25T01:43:21.446446+0000","last_became_peered":"2024-04-25T01:43:21.446446+0000","last_unstale":"2024-04-25T01:44:18.511486+0000","last_undegraded":"2024-04-25T01:44:18.511486+0000","last_fullsized":"2024-04-25T01:44:18.511486+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:21.023845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.510177+0000","last_change":"2024-04-25T01:43:17.027614+0000","last_active":"2024-04-25T01:44:18.510177+0000","last_peered":"2024-04-25T01:44:18.510177+0000","last_clean":"2024-04-25T01:44:18.510177+0000","last_became_active":"2024-04-25T01:43:17.027235+0000","last_became_peered":"2024-04-25T01:43:17.027235+0000","last_unstale":"2024-04-25T01:44:18.510177+0000","last_undegraded":"2024-04-25T01:44:18.510177+0000","last_fullsized":"2024-04-25T01:44:18.510177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:45.627146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.917903+0000","last_change":"2024-04-25T01:43:17.425813+0000","last_active":"2024-04-25T01:44:46.917903+0000","last_peered":"2024-04-25T01:44:46.917903+0000","last_clean":"2024-04-25T01:44:46.917903+0000","last_became_active":"2024-04-25T01:43:17.425469+0000","last_became_peered":"2024-04-25T01:43:17.425469+0000","last_unstale":"2024-04-25T01:44:46.917903+0000","last_undegraded":"2024-04-25T01:44:46.917903+0000","last_fullsized":"2024-04-25T01:44:46.917903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:49:49.863358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.513249+0000","last_change":"2024-04-25T01:43:21.444557+0000","last_active":"2024-04-25T01:44:18.513249+0000","last_peered":"2024-04-25T01:44:18.513249+0000","last_clean":"2024-04-25T01:44:18.513249+0000","last_became_active":"2024-04-25T01:43:21.443809+0000","last_became_peered":"2024-04-25T01:43:21.443809+0000","last_unstale":"2024-04-25T01:44:18.513249+0000","last_undegraded":"2024-04-25T01:44:18.513249+0000","last_fullsized":"2024-04-25T01:44:18.513249+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:29.488448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":110,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:52.523424+0000","last_change":"2024-04-25T01:43:17.032280+0000","last_active":"2024-04-25T01:44:52.523424+0000","last_peered":"2024-04-25T01:44:52.523424+0000","last_clean":"2024-04-25T01:44:52.523424+0000","last_became_active":"2024-04-25T01:43:17.031649+0000","last_became_peered":"2024-04-25T01:43:17.031649+0000","last_unstale":"2024-04-25T01:44:52.523424+0000","last_undegraded":"2024-04-25T01:44:52.523424+0000","last_fullsized":"2024-04-25T01:44:52.523424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:08.532261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.914735+0000","last_change":"2024-04-25T01:44:18.514557+0000","last_active":"2024-04-25T01:44:46.914735+0000","last_peered":"2024-04-25T01:44:46.914735+0000","last_clean":"2024-04-25T01:44:46.914735+0000","last_became_active":"2024-04-25T01:44:18.514194+0000","last_became_peered":"2024-04-25T01:44:18.514194+0000","last_unstale":"2024-04-25T01:44:46.914735+0000","last_undegraded":"2024-04-25T01:44:46.914735+0000","last_fullsized":"2024-04-25T01:44:46.914735+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:19:43.110217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.513117+0000","last_change":"2024-04-25T01:43:21.442683+0000","last_active":"2024-04-25T01:44:18.513117+0000","last_peered":"2024-04-25T01:44:18.513117+0000","last_clean":"2024-04-25T01:44:18.513117+0000","last_became_active":"2024-04-25T01:43:21.442222+0000","last_became_peered":"2024-04-25T01:43:21.442222+0000","last_unstale":"2024-04-25T01:44:18.513117+0000","last_undegraded":"2024-04-25T01:44:18.513117+0000","last_fullsized":"2024-04-25T01:44:18.513117+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:25.196600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.510042+0000","last_change":"2024-04-25T01:43:17.028054+0000","last_active":"2024-04-25T01:44:18.510042+0000","last_peered":"2024-04-25T01:44:18.510042+0000","last_clean":"2024-04-25T01:44:18.510042+0000","last_became_active":"2024-04-25T01:43:17.027639+0000","last_became_peered":"2024-04-25T01:43:17.027639+0000","last_unstale":"2024-04-25T01:44:18.510042+0000","last_undegraded":"2024-04-25T01:44:18.510042+0000","last_fullsized":"2024-04-25T01:44:18.510042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:40.569221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.924130+0000","last_change":"2024-04-25T01:43:17.430417+0000","last_active":"2024-04-25T01:44:46.924130+0000","last_peered":"2024-04-25T01:44:46.924130+0000","last_clean":"2024-04-25T01:44:46.924130+0000","last_became_active":"2024-04-25T01:43:17.430024+0000","last_became_peered":"2024-04-25T01:43:17.430024+0000","last_unstale":"2024-04-25T01:44:46.924130+0000","last_undegraded":"2024-04-25T01:44:46.924130+0000","last_fullsized":"2024-04-25T01:44:46.924130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:56.834530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511592+0000","last_change":"2024-04-25T01:43:21.447732+0000","last_active":"2024-04-25T01:44:18.511592+0000","last_peered":"2024-04-25T01:44:18.511592+0000","last_clean":"2024-04-25T01:44:18.511592+0000","last_became_active":"2024-04-25T01:43:21.447315+0000","last_became_peered":"2024-04-25T01:43:21.447315+0000","last_unstale":"2024-04-25T01:44:18.511592+0000","last_undegraded":"2024-04-25T01:44:18.511592+0000","last_fullsized":"2024-04-25T01:44:18.511592+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:18.138828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511674+0000","last_change":"2024-04-25T01:43:17.018769+0000","last_active":"2024-04-25T01:44:18.511674+0000","last_peered":"2024-04-25T01:44:18.511674+0000","last_clean":"2024-04-25T01:44:18.511674+0000","last_became_active":"2024-04-25T01:43:17.018169+0000","last_became_peered":"2024-04-25T01:43:17.018169+0000","last_unstale":"2024-04-25T01:44:18.511674+0000","last_undegraded":"2024-04-25T01:44:18.511674+0000","last_fullsized":"2024-04-25T01:44:18.511674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:59.112199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.915017+0000","last_change":"2024-04-25T01:43:17.426289+0000","last_active":"2024-04-25T01:44:46.915017+0000","last_peered":"2024-04-25T01:44:46.915017+0000","last_clean":"2024-04-25T01:44:46.915017+0000","last_became_active":"2024-04-25T01:43:17.425512+0000","last_became_peered":"2024-04-25T01:43:17.425512+0000","last_unstale":"2024-04-25T01:44:46.915017+0000","last_undegraded":"2024-04-25T01:44:46.915017+0000","last_fullsized":"2024-04-25T01:44:46.915017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:47.956758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512154+0000","last_change":"2024-04-25T01:43:21.449131+0000","last_active":"2024-04-25T01:44:18.512154+0000","last_peered":"2024-04-25T01:44:18.512154+0000","last_clean":"2024-04-25T01:44:18.512154+0000","last_became_active":"2024-04-25T01:43:21.448752+0000","last_became_peered":"2024-04-25T01:43:21.448752+0000","last_unstale":"2024-04-25T01:44:18.512154+0000","last_undegraded":"2024-04-25T01:44:18.512154+0000","last_fullsized":"2024-04-25T01:44:18.512154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:12:23.109582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.514072+0000","last_change":"2024-04-25T01:43:17.032414+0000","last_active":"2024-04-25T01:44:18.514072+0000","last_peered":"2024-04-25T01:44:18.514072+0000","last_clean":"2024-04-25T01:44:18.514072+0000","last_became_active":"2024-04-25T01:43:17.031848+0000","last_became_peered":"2024-04-25T01:43:17.031848+0000","last_unstale":"2024-04-25T01:44:18.514072+0000","last_undegraded":"2024-04-25T01:44:18.514072+0000","last_fullsized":"2024-04-25T01:44:18.514072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:37.034113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.917038+0000","last_change":"2024-04-25T01:43:17.422831+0000","last_active":"2024-04-25T01:44:46.917038+0000","last_peered":"2024-04-25T01:44:46.917038+0000","last_clean":"2024-04-25T01:44:46.917038+0000","last_became_active":"2024-04-25T01:43:17.422238+0000","last_became_peered":"2024-04-25T01:43:17.422238+0000","last_unstale":"2024-04-25T01:44:46.917038+0000","last_undegraded":"2024-04-25T01:44:46.917038+0000","last_fullsized":"2024-04-25T01:44:46.917038+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:46:05.740054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.580949+0000","last_change":"2024-04-25T01:43:21.442268+0000","last_active":"2024-04-25T01:44:18.580949+0000","last_peered":"2024-04-25T01:44:18.580949+0000","last_clean":"2024-04-25T01:44:18.580949+0000","last_became_active":"2024-04-25T01:43:21.441844+0000","last_became_peered":"2024-04-25T01:43:21.441844+0000","last_unstale":"2024-04-25T01:44:18.580949+0000","last_undegraded":"2024-04-25T01:44:18.580949+0000","last_fullsized":"2024-04-25T01:44:18.580949+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:21.627312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'6","reported_seq":171,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:51.521233+0000","last_change":"2024-04-25T01:43:17.022576+0000","last_active":"2024-04-25T01:44:51.521233+0000","last_peered":"2024-04-25T01:44:51.521233+0000","last_clean":"2024-04-25T01:44:51.521233+0000","last_became_active":"2024-04-25T01:43:17.022223+0000","last_became_peered":"2024-04-25T01:43:17.022223+0000","last_unstale":"2024-04-25T01:44:51.521233+0000","last_undegraded":"2024-04-25T01:44:51.521233+0000","last_fullsized":"2024-04-25T01:44:51.521233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:36:23.905397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.937347+0000","last_change":"2024-04-25T01:43:17.421460+0000","last_active":"2024-04-25T01:44:46.937347+0000","last_peered":"2024-04-25T01:44:46.937347+0000","last_clean":"2024-04-25T01:44:46.937347+0000","last_became_active":"2024-04-25T01:43:17.421153+0000","last_became_peered":"2024-04-25T01:43:17.421153+0000","last_unstale":"2024-04-25T01:44:46.937347+0000","last_undegraded":"2024-04-25T01:44:46.937347+0000","last_fullsized":"2024-04-25T01:44:46.937347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:50.285476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509118+0000","last_change":"2024-04-25T01:43:21.450961+0000","last_active":"2024-04-25T01:44:18.509118+0000","last_peered":"2024-04-25T01:44:18.509118+0000","last_clean":"2024-04-25T01:44:18.509118+0000","last_became_active":"2024-04-25T01:43:21.450578+0000","last_became_peered":"2024-04-25T01:43:21.450578+0000","last_unstale":"2024-04-25T01:44:18.509118+0000","last_undegraded":"2024-04-25T01:44:18.509118+0000","last_fullsized":"2024-04-25T01:44:18.509118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:04.222443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512764+0000","last_change":"2024-04-25T01:43:17.030973+0000","last_active":"2024-04-25T01:44:18.512764+0000","last_peered":"2024-04-25T01:44:18.512764+0000","last_clean":"2024-04-25T01:44:18.512764+0000","last_became_active":"2024-04-25T01:43:17.030458+0000","last_became_peered":"2024-04-25T01:43:17.030458+0000","last_unstale":"2024-04-25T01:44:18.512764+0000","last_undegraded":"2024-04-25T01:44:18.512764+0000","last_fullsized":"2024-04-25T01:44:18.512764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:19.187667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.921223+0000","last_change":"2024-04-25T01:44:18.517319+0000","last_active":"2024-04-25T01:44:46.921223+0000","last_peered":"2024-04-25T01:44:46.921223+0000","last_clean":"2024-04-25T01:44:46.921223+0000","last_became_active":"2024-04-25T01:44:18.517046+0000","last_became_peered":"2024-04-25T01:44:18.517046+0000","last_unstale":"2024-04-25T01:44:46.921223+0000","last_undegraded":"2024-04-25T01:44:46.921223+0000","last_fullsized":"2024-04-25T01:44:46.921223+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:10.066856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512411+0000","last_change":"2024-04-25T01:43:21.452287+0000","last_active":"2024-04-25T01:44:18.512411+0000","last_peered":"2024-04-25T01:44:18.512411+0000","last_clean":"2024-04-25T01:44:18.512411+0000","last_became_active":"2024-04-25T01:43:21.451765+0000","last_became_peered":"2024-04-25T01:43:21.451765+0000","last_unstale":"2024-04-25T01:44:18.512411+0000","last_undegraded":"2024-04-25T01:44:18.512411+0000","last_fullsized":"2024-04-25T01:44:18.512411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:44:02.353645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.515621+0000","last_change":"2024-04-25T01:43:17.022319+0000","last_active":"2024-04-25T01:44:18.515621+0000","last_peered":"2024-04-25T01:44:18.515621+0000","last_clean":"2024-04-25T01:44:18.515621+0000","last_became_active":"2024-04-25T01:43:17.021944+0000","last_became_peered":"2024-04-25T01:43:17.021944+0000","last_unstale":"2024-04-25T01:44:18.515621+0000","last_undegraded":"2024-04-25T01:44:18.515621+0000","last_fullsized":"2024-04-25T01:44:18.515621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:42.856997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.938103+0000","last_change":"2024-04-25T01:43:17.418164+0000","last_active":"2024-04-25T01:44:46.938103+0000","last_peered":"2024-04-25T01:44:46.938103+0000","last_clean":"2024-04-25T01:44:46.938103+0000","last_became_active":"2024-04-25T01:43:17.417846+0000","last_became_peered":"2024-04-25T01:43:17.417846+0000","last_unstale":"2024-04-25T01:44:46.938103+0000","last_undegraded":"2024-04-25T01:44:46.938103+0000","last_fullsized":"2024-04-25T01:44:46.938103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:05:25.973457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509285+0000","last_change":"2024-04-25T01:43:21.443429+0000","last_active":"2024-04-25T01:44:18.509285+0000","last_peered":"2024-04-25T01:44:18.509285+0000","last_clean":"2024-04-25T01:44:18.509285+0000","last_became_active":"2024-04-25T01:43:21.442210+0000","last_became_peered":"2024-04-25T01:43:21.442210+0000","last_unstale":"2024-04-25T01:44:18.509285+0000","last_undegraded":"2024-04-25T01:44:18.509285+0000","last_fullsized":"2024-04-25T01:44:18.509285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:29.578740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'37","reported_seq":213,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:51.530150+0000","last_change":"2024-04-25T01:43:17.020501+0000","last_active":"2024-04-25T01:44:51.530150+0000","last_peered":"2024-04-25T01:44:51.530150+0000","last_clean":"2024-04-25T01:44:51.530150+0000","last_became_active":"2024-04-25T01:43:17.020160+0000","last_became_peered":"2024-04-25T01:43:17.020160+0000","last_unstale":"2024-04-25T01:44:51.530150+0000","last_undegraded":"2024-04-25T01:44:51.530150+0000","last_fullsized":"2024-04-25T01:44:51.530150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:49.273381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":137,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.915575+0000","last_change":"2024-04-25T01:43:17.426895+0000","last_active":"2024-04-25T01:44:46.915575+0000","last_peered":"2024-04-25T01:44:46.915575+0000","last_clean":"2024-04-25T01:44:46.915575+0000","last_became_active":"2024-04-25T01:43:17.426572+0000","last_became_peered":"2024-04-25T01:43:17.426572+0000","last_unstale":"2024-04-25T01:44:46.915575+0000","last_undegraded":"2024-04-25T01:44:46.915575+0000","last_fullsized":"2024-04-25T01:44:46.915575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:36.166282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.515916+0000","last_change":"2024-04-25T01:43:21.442204+0000","last_active":"2024-04-25T01:44:18.515916+0000","last_peered":"2024-04-25T01:44:18.515916+0000","last_clean":"2024-04-25T01:44:18.515916+0000","last_became_active":"2024-04-25T01:43:21.441779+0000","last_became_peered":"2024-04-25T01:43:21.441779+0000","last_unstale":"2024-04-25T01:44:18.515916+0000","last_undegraded":"2024-04-25T01:44:18.515916+0000","last_fullsized":"2024-04-25T01:44:18.515916+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:12.075657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.513718+0000","last_change":"2024-04-25T01:43:17.028737+0000","last_active":"2024-04-25T01:44:18.513718+0000","last_peered":"2024-04-25T01:44:18.513718+0000","last_clean":"2024-04-25T01:44:18.513718+0000","last_became_active":"2024-04-25T01:43:17.028349+0000","last_became_peered":"2024-04-25T01:43:17.028349+0000","last_unstale":"2024-04-25T01:44:18.513718+0000","last_undegraded":"2024-04-25T01:44:18.513718+0000","last_fullsized":"2024-04-25T01:44:18.513718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:04.802232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.915856+0000","last_change":"2024-04-25T01:43:17.436315+0000","last_active":"2024-04-25T01:44:46.915856+0000","last_peered":"2024-04-25T01:44:46.915856+0000","last_clean":"2024-04-25T01:44:46.915856+0000","last_became_active":"2024-04-25T01:43:17.436007+0000","last_became_peered":"2024-04-25T01:43:17.436007+0000","last_unstale":"2024-04-25T01:44:46.915856+0000","last_undegraded":"2024-04-25T01:44:46.915856+0000","last_fullsized":"2024-04-25T01:44:46.915856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:49.091092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.518839+0000","last_change":"2024-04-25T01:44:18.518839+0000","last_active":"2024-04-25T01:44:18.518839+0000","last_peered":"2024-04-25T01:44:18.518839+0000","last_clean":"2024-04-25T01:44:18.518839+0000","last_became_active":"2024-04-25T01:44:18.518237+0000","last_became_peered":"2024-04-25T01:44:18.518237+0000","last_unstale":"2024-04-25T01:44:18.518839+0000","last_undegraded":"2024-04-25T01:44:18.518839+0000","last_fullsized":"2024-04-25T01:44:18.518839+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:51.963839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.513983+0000","last_change":"2024-04-25T01:43:17.029513+0000","last_active":"2024-04-25T01:44:18.513983+0000","last_peered":"2024-04-25T01:44:18.513983+0000","last_clean":"2024-04-25T01:44:18.513983+0000","last_became_active":"2024-04-25T01:43:17.029203+0000","last_became_peered":"2024-04-25T01:43:17.029203+0000","last_unstale":"2024-04-25T01:44:18.513983+0000","last_undegraded":"2024-04-25T01:44:18.513983+0000","last_fullsized":"2024-04-25T01:44:18.513983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:47.002587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.929759+0000","last_change":"2024-04-25T01:43:17.435273+0000","last_active":"2024-04-25T01:44:46.929759+0000","last_peered":"2024-04-25T01:44:46.929759+0000","last_clean":"2024-04-25T01:44:46.929759+0000","last_became_active":"2024-04-25T01:43:17.434892+0000","last_became_peered":"2024-04-25T01:43:17.434892+0000","last_unstale":"2024-04-25T01:44:46.929759+0000","last_undegraded":"2024-04-25T01:44:46.929759+0000","last_fullsized":"2024-04-25T01:44:46.929759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:55.342440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509673+0000","last_change":"2024-04-25T01:43:21.448363+0000","last_active":"2024-04-25T01:44:18.509673+0000","last_peered":"2024-04-25T01:44:18.509673+0000","last_clean":"2024-04-25T01:44:18.509673+0000","last_became_active":"2024-04-25T01:43:21.445980+0000","last_became_peered":"2024-04-25T01:43:21.445980+0000","last_unstale":"2024-04-25T01:44:18.509673+0000","last_undegraded":"2024-04-25T01:44:18.509673+0000","last_fullsized":"2024-04-25T01:44:18.509673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:05.890579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":110,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:52.532172+0000","last_change":"2024-04-25T01:43:17.028557+0000","last_active":"2024-04-25T01:44:52.532172+0000","last_peered":"2024-04-25T01:44:52.532172+0000","last_clean":"2024-04-25T01:44:52.532172+0000","last_became_active":"2024-04-25T01:43:17.028199+0000","last_became_peered":"2024-04-25T01:43:17.028199+0000","last_unstale":"2024-04-25T01:44:52.532172+0000","last_undegraded":"2024-04-25T01:44:52.532172+0000","last_fullsized":"2024-04-25T01:44:52.532172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:41.809220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.926494+0000","last_change":"2024-04-25T01:43:17.437002+0000","last_active":"2024-04-25T01:44:46.926494+0000","last_peered":"2024-04-25T01:44:46.926494+0000","last_clean":"2024-04-25T01:44:46.926494+0000","last_became_active":"2024-04-25T01:43:17.436509+0000","last_became_peered":"2024-04-25T01:43:17.436509+0000","last_unstale":"2024-04-25T01:44:46.926494+0000","last_undegraded":"2024-04-25T01:44:46.926494+0000","last_fullsized":"2024-04-25T01:44:46.926494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:49.329249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.582692+0000","last_change":"2024-04-25T01:44:18.582692+0000","last_active":"2024-04-25T01:44:18.582692+0000","last_peered":"2024-04-25T01:44:18.582692+0000","last_clean":"2024-04-25T01:44:18.582692+0000","last_became_active":"2024-04-25T01:44:18.582271+0000","last_became_peered":"2024-04-25T01:44:18.582271+0000","last_unstale":"2024-04-25T01:44:18.582692+0000","last_undegraded":"2024-04-25T01:44:18.582692+0000","last_fullsized":"2024-04-25T01:44:18.582692+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:56.946759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509402+0000","last_change":"2024-04-25T01:43:17.028185+0000","last_active":"2024-04-25T01:44:18.509402+0000","last_peered":"2024-04-25T01:44:18.509402+0000","last_clean":"2024-04-25T01:44:18.509402+0000","last_became_active":"2024-04-25T01:43:17.027851+0000","last_became_peered":"2024-04-25T01:43:17.027851+0000","last_unstale":"2024-04-25T01:44:18.509402+0000","last_undegraded":"2024-04-25T01:44:18.509402+0000","last_fullsized":"2024-04-25T01:44:18.509402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:02.320793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.918096+0000","last_change":"2024-04-25T01:43:17.424990+0000","last_active":"2024-04-25T01:44:46.918096+0000","last_peered":"2024-04-25T01:44:46.918096+0000","last_clean":"2024-04-25T01:44:46.918096+0000","last_became_active":"2024-04-25T01:43:17.424598+0000","last_became_peered":"2024-04-25T01:43:17.424598+0000","last_unstale":"2024-04-25T01:44:46.918096+0000","last_undegraded":"2024-04-25T01:44:46.918096+0000","last_fullsized":"2024-04-25T01:44:46.918096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:34.855089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.514597+0000","last_change":"2024-04-25T01:43:21.444138+0000","last_active":"2024-04-25T01:44:18.514597+0000","last_peered":"2024-04-25T01:44:18.514597+0000","last_clean":"2024-04-25T01:44:18.514597+0000","last_became_active":"2024-04-25T01:43:21.443283+0000","last_became_peered":"2024-04-25T01:43:21.443283+0000","last_unstale":"2024-04-25T01:44:18.514597+0000","last_undegraded":"2024-04-25T01:44:18.514597+0000","last_fullsized":"2024-04-25T01:44:18.514597+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:46.275789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.510471+0000","last_change":"2024-04-25T01:43:17.027892+0000","last_active":"2024-04-25T01:44:18.510471+0000","last_peered":"2024-04-25T01:44:18.510471+0000","last_clean":"2024-04-25T01:44:18.510471+0000","last_became_active":"2024-04-25T01:43:17.027476+0000","last_became_peered":"2024-04-25T01:43:17.027476+0000","last_unstale":"2024-04-25T01:44:18.510471+0000","last_undegraded":"2024-04-25T01:44:18.510471+0000","last_fullsized":"2024-04-25T01:44:18.510471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:31.079700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.924320+0000","last_change":"2024-04-25T01:43:17.431491+0000","last_active":"2024-04-25T01:44:46.924320+0000","last_peered":"2024-04-25T01:44:46.924320+0000","last_clean":"2024-04-25T01:44:46.924320+0000","last_became_active":"2024-04-25T01:43:17.431160+0000","last_became_peered":"2024-04-25T01:43:17.431160+0000","last_unstale":"2024-04-25T01:44:46.924320+0000","last_undegraded":"2024-04-25T01:44:46.924320+0000","last_fullsized":"2024-04-25T01:44:46.924320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:03.612857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509454+0000","last_change":"2024-04-25T01:43:21.626398+0000","last_active":"2024-04-25T01:44:18.509454+0000","last_peered":"2024-04-25T01:44:18.509454+0000","last_clean":"2024-04-25T01:44:18.509454+0000","last_became_active":"2024-04-25T01:43:21.625856+0000","last_became_peered":"2024-04-25T01:43:21.625856+0000","last_unstale":"2024-04-25T01:44:18.509454+0000","last_undegraded":"2024-04-25T01:44:18.509454+0000","last_fullsized":"2024-04-25T01:44:18.509454+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:53.000912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509470+0000","last_change":"2024-04-25T01:43:17.027564+0000","last_active":"2024-04-25T01:44:18.509470+0000","last_peered":"2024-04-25T01:44:18.509470+0000","last_clean":"2024-04-25T01:44:18.509470+0000","last_became_active":"2024-04-25T01:43:17.027218+0000","last_became_peered":"2024-04-25T01:43:17.027218+0000","last_unstale":"2024-04-25T01:44:18.509470+0000","last_undegraded":"2024-04-25T01:44:18.509470+0000","last_fullsized":"2024-04-25T01:44:18.509470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:52.658023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.914564+0000","last_change":"2024-04-25T01:43:17.431647+0000","last_active":"2024-04-25T01:44:46.914564+0000","last_peered":"2024-04-25T01:44:46.914564+0000","last_clean":"2024-04-25T01:44:46.914564+0000","last_became_active":"2024-04-25T01:43:17.431048+0000","last_became_peered":"2024-04-25T01:43:17.431048+0000","last_unstale":"2024-04-25T01:44:46.914564+0000","last_undegraded":"2024-04-25T01:44:46.914564+0000","last_fullsized":"2024-04-25T01:44:46.914564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:30.124585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512606+0000","last_change":"2024-04-25T01:43:21.450326+0000","last_active":"2024-04-25T01:44:18.512606+0000","last_peered":"2024-04-25T01:44:18.512606+0000","last_clean":"2024-04-25T01:44:18.512606+0000","last_became_active":"2024-04-25T01:43:21.449925+0000","last_became_peered":"2024-04-25T01:43:21.449925+0000","last_unstale":"2024-04-25T01:44:18.512606+0000","last_undegraded":"2024-04-25T01:44:18.512606+0000","last_fullsized":"2024-04-25T01:44:18.512606+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:39.896905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":102,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:51.532717+0000","last_change":"2024-04-25T01:43:17.032223+0000","last_active":"2024-04-25T01:44:51.532717+0000","last_peered":"2024-04-25T01:44:51.532717+0000","last_clean":"2024-04-25T01:44:51.532717+0000","last_became_active":"2024-04-25T01:43:17.031774+0000","last_became_peered":"2024-04-25T01:43:17.031774+0000","last_unstale":"2024-04-25T01:44:51.532717+0000","last_undegraded":"2024-04-25T01:44:51.532717+0000","last_fullsized":"2024-04-25T01:44:51.532717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:51:03.730850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.922359+0000","last_change":"2024-04-25T01:43:17.425756+0000","last_active":"2024-04-25T01:44:46.922359+0000","last_peered":"2024-04-25T01:44:46.922359+0000","last_clean":"2024-04-25T01:44:46.922359+0000","last_became_active":"2024-04-25T01:43:17.425433+0000","last_became_peered":"2024-04-25T01:43:17.425433+0000","last_unstale":"2024-04-25T01:44:46.922359+0000","last_undegraded":"2024-04-25T01:44:46.922359+0000","last_fullsized":"2024-04-25T01:44:46.922359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:29.776771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511889+0000","last_change":"2024-04-25T01:43:21.625898+0000","last_active":"2024-04-25T01:44:18.511889+0000","last_peered":"2024-04-25T01:44:18.511889+0000","last_clean":"2024-04-25T01:44:18.511889+0000","last_became_active":"2024-04-25T01:43:21.625460+0000","last_became_peered":"2024-04-25T01:43:21.625460+0000","last_unstale":"2024-04-25T01:44:18.511889+0000","last_undegraded":"2024-04-25T01:44:18.511889+0000","last_fullsized":"2024-04-25T01:44:18.511889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:31.199172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.906587+0000","last_change":"2024-04-25T01:44:18.515838+0000","last_active":"2024-04-25T01:44:46.906587+0000","last_peered":"2024-04-25T01:44:46.906587+0000","last_clean":"2024-04-25T01:44:46.906587+0000","last_became_active":"2024-04-25T01:44:18.515440+0000","last_became_peered":"2024-04-25T01:44:18.515440+0000","last_unstale":"2024-04-25T01:44:46.906587+0000","last_undegraded":"2024-04-25T01:44:46.906587+0000","last_fullsized":"2024-04-25T01:44:46.906587+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:54:33.072084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:52.531524+0000","last_change":"2024-04-25T01:43:17.028904+0000","last_active":"2024-04-25T01:44:52.531524+0000","last_peered":"2024-04-25T01:44:52.531524+0000","last_clean":"2024-04-25T01:44:52.531524+0000","last_became_active":"2024-04-25T01:43:17.028608+0000","last_became_peered":"2024-04-25T01:43:17.028608+0000","last_unstale":"2024-04-25T01:44:52.531524+0000","last_undegraded":"2024-04-25T01:44:52.531524+0000","last_fullsized":"2024-04-25T01:44:52.531524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:19.297926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.521409+0000","last_change":"2024-04-25T01:44:18.521409+0000","last_active":"2024-04-25T01:44:18.521409+0000","last_peered":"2024-04-25T01:44:18.521409+0000","last_clean":"2024-04-25T01:44:18.521409+0000","last_became_active":"2024-04-25T01:44:18.515062+0000","last_became_peered":"2024-04-25T01:44:18.515062+0000","last_unstale":"2024-04-25T01:44:18.521409+0000","last_undegraded":"2024-04-25T01:44:18.521409+0000","last_fullsized":"2024-04-25T01:44:18.521409+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:09:30.072124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10,11,9],"acting":[7,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919567+0000","last_change":"2024-04-25T01:43:17.428636+0000","last_active":"2024-04-25T01:44:46.919567+0000","last_peered":"2024-04-25T01:44:46.919567+0000","last_clean":"2024-04-25T01:44:46.919567+0000","last_became_active":"2024-04-25T01:43:17.428316+0000","last_became_peered":"2024-04-25T01:43:17.428316+0000","last_unstale":"2024-04-25T01:44:46.919567+0000","last_undegraded":"2024-04-25T01:44:46.919567+0000","last_fullsized":"2024-04-25T01:44:46.919567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:04.683846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512267+0000","last_change":"2024-04-25T01:43:17.020632+0000","last_active":"2024-04-25T01:44:18.512267+0000","last_peered":"2024-04-25T01:44:18.512267+0000","last_clean":"2024-04-25T01:44:18.512267+0000","last_became_active":"2024-04-25T01:43:17.020278+0000","last_became_peered":"2024-04-25T01:43:17.020278+0000","last_unstale":"2024-04-25T01:44:18.512267+0000","last_undegraded":"2024-04-25T01:44:18.512267+0000","last_fullsized":"2024-04-25T01:44:18.512267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:13.232470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512749+0000","last_change":"2024-04-25T01:42:05.003964+0000","last_active":"2024-04-25T01:44:18.512749+0000","last_peered":"2024-04-25T01:44:18.512749+0000","last_clean":"2024-04-25T01:44:18.512749+0000","last_became_active":"2024-04-25T01:42:05.003509+0000","last_became_peered":"2024-04-25T01:42:05.003509+0000","last_unstale":"2024-04-25T01:44:18.512749+0000","last_undegraded":"2024-04-25T01:44:18.512749+0000","last_fullsized":"2024-04-25T01:44:18.512749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:38:39.137002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:38:39.137002+0000","last_clean_scrub_stamp":"2024-04-25T01:38:39.137002+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:53.878838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.515627+0000","last_change":"2024-04-25T01:43:21.626345+0000","last_active":"2024-04-25T01:44:18.515627+0000","last_peered":"2024-04-25T01:44:18.515627+0000","last_clean":"2024-04-25T01:44:18.515627+0000","last_became_active":"2024-04-25T01:43:21.625884+0000","last_became_peered":"2024-04-25T01:43:21.625884+0000","last_unstale":"2024-04-25T01:44:18.515627+0000","last_undegraded":"2024-04-25T01:44:18.515627+0000","last_fullsized":"2024-04-25T01:44:18.515627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:37.961617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.916916+0000","last_change":"2024-04-25T01:43:17.421186+0000","last_active":"2024-04-25T01:44:46.916916+0000","last_peered":"2024-04-25T01:44:46.916916+0000","last_clean":"2024-04-25T01:44:46.916916+0000","last_became_active":"2024-04-25T01:43:17.420824+0000","last_became_peered":"2024-04-25T01:43:17.420824+0000","last_unstale":"2024-04-25T01:44:46.916916+0000","last_undegraded":"2024-04-25T01:44:46.916916+0000","last_fullsized":"2024-04-25T01:44:46.916916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:27.829126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512196+0000","last_change":"2024-04-25T01:43:17.034050+0000","last_active":"2024-04-25T01:44:18.512196+0000","last_peered":"2024-04-25T01:44:18.512196+0000","last_clean":"2024-04-25T01:44:18.512196+0000","last_became_active":"2024-04-25T01:43:17.033725+0000","last_became_peered":"2024-04-25T01:43:17.033725+0000","last_unstale":"2024-04-25T01:44:18.512196+0000","last_undegraded":"2024-04-25T01:44:18.512196+0000","last_fullsized":"2024-04-25T01:44:18.512196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:06.419265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511329+0000","last_change":"2024-04-25T01:43:21.443299+0000","last_active":"2024-04-25T01:44:18.511329+0000","last_peered":"2024-04-25T01:44:18.511329+0000","last_clean":"2024-04-25T01:44:18.511329+0000","last_became_active":"2024-04-25T01:43:21.442899+0000","last_became_peered":"2024-04-25T01:43:21.442899+0000","last_unstale":"2024-04-25T01:44:18.511329+0000","last_undegraded":"2024-04-25T01:44:18.511329+0000","last_fullsized":"2024-04-25T01:44:18.511329+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:11:51.720208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509960+0000","last_change":"2024-04-25T01:43:17.035136+0000","last_active":"2024-04-25T01:44:18.509960+0000","last_peered":"2024-04-25T01:44:18.509960+0000","last_clean":"2024-04-25T01:44:18.509960+0000","last_became_active":"2024-04-25T01:43:17.034289+0000","last_became_peered":"2024-04-25T01:43:17.034289+0000","last_unstale":"2024-04-25T01:44:18.509960+0000","last_undegraded":"2024-04-25T01:44:18.509960+0000","last_fullsized":"2024-04-25T01:44:18.509960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:07.450025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.908477+0000","last_change":"2024-04-25T01:43:17.414478+0000","last_active":"2024-04-25T01:44:46.908477+0000","last_peered":"2024-04-25T01:44:46.908477+0000","last_clean":"2024-04-25T01:44:46.908477+0000","last_became_active":"2024-04-25T01:43:17.414061+0000","last_became_peered":"2024-04-25T01:43:17.414061+0000","last_unstale":"2024-04-25T01:44:46.908477+0000","last_undegraded":"2024-04-25T01:44:46.908477+0000","last_fullsized":"2024-04-25T01:44:46.908477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:16.051518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.581417+0000","last_change":"2024-04-25T01:43:21.625963+0000","last_active":"2024-04-25T01:44:18.581417+0000","last_peered":"2024-04-25T01:44:18.581417+0000","last_clean":"2024-04-25T01:44:18.581417+0000","last_became_active":"2024-04-25T01:43:21.625563+0000","last_became_peered":"2024-04-25T01:43:21.625563+0000","last_unstale":"2024-04-25T01:44:18.581417+0000","last_undegraded":"2024-04-25T01:44:18.581417+0000","last_fullsized":"2024-04-25T01:44:18.581417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:11.217504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509914+0000","last_change":"2024-04-25T01:43:17.024391+0000","last_active":"2024-04-25T01:44:18.509914+0000","last_peered":"2024-04-25T01:44:18.509914+0000","last_clean":"2024-04-25T01:44:18.509914+0000","last_became_active":"2024-04-25T01:43:17.023994+0000","last_became_peered":"2024-04-25T01:43:17.023994+0000","last_unstale":"2024-04-25T01:44:18.509914+0000","last_undegraded":"2024-04-25T01:44:18.509914+0000","last_fullsized":"2024-04-25T01:44:18.509914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:56.489383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.914837+0000","last_change":"2024-04-25T01:43:17.435357+0000","last_active":"2024-04-25T01:44:46.914837+0000","last_peered":"2024-04-25T01:44:46.914837+0000","last_clean":"2024-04-25T01:44:46.914837+0000","last_became_active":"2024-04-25T01:43:17.435090+0000","last_became_peered":"2024-04-25T01:43:17.435090+0000","last_unstale":"2024-04-25T01:44:46.914837+0000","last_undegraded":"2024-04-25T01:44:46.914837+0000","last_fullsized":"2024-04-25T01:44:46.914837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:49.181850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.515185+0000","last_change":"2024-04-25T01:43:21.441314+0000","last_active":"2024-04-25T01:44:18.515185+0000","last_peered":"2024-04-25T01:44:18.515185+0000","last_clean":"2024-04-25T01:44:18.515185+0000","last_became_active":"2024-04-25T01:43:21.440896+0000","last_became_peered":"2024-04-25T01:43:21.440896+0000","last_unstale":"2024-04-25T01:44:18.515185+0000","last_undegraded":"2024-04-25T01:44:18.515185+0000","last_fullsized":"2024-04-25T01:44:18.515185+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:04:36.091128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512284+0000","last_change":"2024-04-25T01:43:17.031525+0000","last_active":"2024-04-25T01:44:18.512284+0000","last_peered":"2024-04-25T01:44:18.512284+0000","last_clean":"2024-04-25T01:44:18.512284+0000","last_became_active":"2024-04-25T01:43:17.031203+0000","last_became_peered":"2024-04-25T01:43:17.031203+0000","last_unstale":"2024-04-25T01:44:18.512284+0000","last_undegraded":"2024-04-25T01:44:18.512284+0000","last_fullsized":"2024-04-25T01:44:18.512284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:22.929179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919567+0000","last_change":"2024-04-25T01:43:17.425981+0000","last_active":"2024-04-25T01:44:46.919567+0000","last_peered":"2024-04-25T01:44:46.919567+0000","last_clean":"2024-04-25T01:44:46.919567+0000","last_became_active":"2024-04-25T01:43:17.425674+0000","last_became_peered":"2024-04-25T01:43:17.425674+0000","last_unstale":"2024-04-25T01:44:46.919567+0000","last_undegraded":"2024-04-25T01:44:46.919567+0000","last_fullsized":"2024-04-25T01:44:46.919567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:58.441774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511438+0000","last_change":"2024-04-25T01:43:21.626389+0000","last_active":"2024-04-25T01:44:18.511438+0000","last_peered":"2024-04-25T01:44:18.511438+0000","last_clean":"2024-04-25T01:44:18.511438+0000","last_became_active":"2024-04-25T01:43:21.625859+0000","last_became_peered":"2024-04-25T01:43:21.625859+0000","last_unstale":"2024-04-25T01:44:18.511438+0000","last_undegraded":"2024-04-25T01:44:18.511438+0000","last_fullsized":"2024-04-25T01:44:18.511438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:51.372126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.730959+0000","last_change":"2024-04-25T01:43:17.022903+0000","last_active":"2024-04-25T01:44:18.730959+0000","last_peered":"2024-04-25T01:44:18.730959+0000","last_clean":"2024-04-25T01:44:18.730959+0000","last_became_active":"2024-04-25T01:43:17.022564+0000","last_became_peered":"2024-04-25T01:43:17.022564+0000","last_unstale":"2024-04-25T01:44:18.730959+0000","last_undegraded":"2024-04-25T01:44:18.730959+0000","last_fullsized":"2024-04-25T01:44:18.730959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:01.277500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.906202+0000","last_change":"2024-04-25T01:43:17.422855+0000","last_active":"2024-04-25T01:44:46.906202+0000","last_peered":"2024-04-25T01:44:46.906202+0000","last_clean":"2024-04-25T01:44:46.906202+0000","last_became_active":"2024-04-25T01:43:17.422490+0000","last_became_peered":"2024-04-25T01:43:17.422490+0000","last_unstale":"2024-04-25T01:44:46.906202+0000","last_undegraded":"2024-04-25T01:44:46.906202+0000","last_fullsized":"2024-04-25T01:44:46.906202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:40.016044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509793+0000","last_change":"2024-04-25T01:43:21.440080+0000","last_active":"2024-04-25T01:44:18.509793+0000","last_peered":"2024-04-25T01:44:18.509793+0000","last_clean":"2024-04-25T01:44:18.509793+0000","last_became_active":"2024-04-25T01:43:21.439669+0000","last_became_peered":"2024-04-25T01:43:21.439669+0000","last_unstale":"2024-04-25T01:44:18.509793+0000","last_undegraded":"2024-04-25T01:44:18.509793+0000","last_fullsized":"2024-04-25T01:44:18.509793+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:34.143210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.581111+0000","last_change":"2024-04-25T01:43:17.035521+0000","last_active":"2024-04-25T01:44:18.581111+0000","last_peered":"2024-04-25T01:44:18.581111+0000","last_clean":"2024-04-25T01:44:18.581111+0000","last_became_active":"2024-04-25T01:43:17.033762+0000","last_became_peered":"2024-04-25T01:43:17.033762+0000","last_unstale":"2024-04-25T01:44:18.581111+0000","last_undegraded":"2024-04-25T01:44:18.581111+0000","last_fullsized":"2024-04-25T01:44:18.581111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:03:04.762578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.922112+0000","last_change":"2024-04-25T01:43:17.426952+0000","last_active":"2024-04-25T01:44:46.922112+0000","last_peered":"2024-04-25T01:44:46.922112+0000","last_clean":"2024-04-25T01:44:46.922112+0000","last_became_active":"2024-04-25T01:43:17.426573+0000","last_became_peered":"2024-04-25T01:43:17.426573+0000","last_unstale":"2024-04-25T01:44:46.922112+0000","last_undegraded":"2024-04-25T01:44:46.922112+0000","last_fullsized":"2024-04-25T01:44:46.922112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:33.807254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511797+0000","last_change":"2024-04-25T01:43:21.453295+0000","last_active":"2024-04-25T01:44:18.511797+0000","last_peered":"2024-04-25T01:44:18.511797+0000","last_clean":"2024-04-25T01:44:18.511797+0000","last_became_active":"2024-04-25T01:43:21.452031+0000","last_became_peered":"2024-04-25T01:43:21.452031+0000","last_unstale":"2024-04-25T01:44:18.511797+0000","last_undegraded":"2024-04-25T01:44:18.511797+0000","last_fullsized":"2024-04-25T01:44:18.511797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:23.956638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.515227+0000","last_change":"2024-04-25T01:43:17.029253+0000","last_active":"2024-04-25T01:44:18.515227+0000","last_peered":"2024-04-25T01:44:18.515227+0000","last_clean":"2024-04-25T01:44:18.515227+0000","last_became_active":"2024-04-25T01:43:17.028949+0000","last_became_peered":"2024-04-25T01:43:17.028949+0000","last_unstale":"2024-04-25T01:44:18.515227+0000","last_undegraded":"2024-04-25T01:44:18.515227+0000","last_fullsized":"2024-04-25T01:44:18.515227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:05.101916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.920902+0000","last_change":"2024-04-25T01:43:17.431319+0000","last_active":"2024-04-25T01:44:46.920902+0000","last_peered":"2024-04-25T01:44:46.920902+0000","last_clean":"2024-04-25T01:44:46.920902+0000","last_became_active":"2024-04-25T01:43:17.431007+0000","last_became_peered":"2024-04-25T01:43:17.431007+0000","last_unstale":"2024-04-25T01:44:46.920902+0000","last_undegraded":"2024-04-25T01:44:46.920902+0000","last_fullsized":"2024-04-25T01:44:46.920902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:38.999286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511443+0000","last_change":"2024-04-25T01:43:21.625931+0000","last_active":"2024-04-25T01:44:18.511443+0000","last_peered":"2024-04-25T01:44:18.511443+0000","last_clean":"2024-04-25T01:44:18.511443+0000","last_became_active":"2024-04-25T01:43:21.625460+0000","last_became_peered":"2024-04-25T01:43:21.625460+0000","last_unstale":"2024-04-25T01:44:18.511443+0000","last_undegraded":"2024-04-25T01:44:18.511443+0000","last_fullsized":"2024-04-25T01:44:18.511443+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:59.665909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.509835+0000","last_change":"2024-04-25T01:43:17.033937+0000","last_active":"2024-04-25T01:44:18.509835+0000","last_peered":"2024-04-25T01:44:18.509835+0000","last_clean":"2024-04-25T01:44:18.509835+0000","last_became_active":"2024-04-25T01:43:17.033629+0000","last_became_peered":"2024-04-25T01:43:17.033629+0000","last_unstale":"2024-04-25T01:44:18.509835+0000","last_undegraded":"2024-04-25T01:44:18.509835+0000","last_fullsized":"2024-04-25T01:44:18.509835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:05.531624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.936394+0000","last_change":"2024-04-25T01:43:17.436068+0000","last_active":"2024-04-25T01:44:46.936394+0000","last_peered":"2024-04-25T01:44:46.936394+0000","last_clean":"2024-04-25T01:44:46.936394+0000","last_became_active":"2024-04-25T01:43:17.435383+0000","last_became_peered":"2024-04-25T01:43:17.435383+0000","last_unstale":"2024-04-25T01:44:46.936394+0000","last_undegraded":"2024-04-25T01:44:46.936394+0000","last_fullsized":"2024-04-25T01:44:46.936394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:36:00.087923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.730978+0000","last_change":"2024-04-25T01:43:21.441696+0000","last_active":"2024-04-25T01:44:18.730978+0000","last_peered":"2024-04-25T01:44:18.730978+0000","last_clean":"2024-04-25T01:44:18.730978+0000","last_became_active":"2024-04-25T01:43:21.441263+0000","last_became_peered":"2024-04-25T01:43:21.441263+0000","last_unstale":"2024-04-25T01:44:18.730978+0000","last_undegraded":"2024-04-25T01:44:18.730978+0000","last_fullsized":"2024-04-25T01:44:18.730978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:47:57.263465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":107,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:52.550772+0000","last_change":"2024-04-25T01:43:17.033841+0000","last_active":"2024-04-25T01:44:52.550772+0000","last_peered":"2024-04-25T01:44:52.550772+0000","last_clean":"2024-04-25T01:44:52.550772+0000","last_became_active":"2024-04-25T01:43:17.033477+0000","last_became_peered":"2024-04-25T01:43:17.033477+0000","last_unstale":"2024-04-25T01:44:52.550772+0000","last_undegraded":"2024-04-25T01:44:52.550772+0000","last_fullsized":"2024-04-25T01:44:52.550772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:43.717525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.936319+0000","last_change":"2024-04-25T01:43:17.430744+0000","last_active":"2024-04-25T01:44:46.936319+0000","last_peered":"2024-04-25T01:44:46.936319+0000","last_clean":"2024-04-25T01:44:46.936319+0000","last_became_active":"2024-04-25T01:43:17.430434+0000","last_became_peered":"2024-04-25T01:43:17.430434+0000","last_unstale":"2024-04-25T01:44:46.936319+0000","last_undegraded":"2024-04-25T01:44:46.936319+0000","last_fullsized":"2024-04-25T01:44:46.936319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:17.895901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.511855+0000","last_change":"2024-04-25T01:43:21.451604+0000","last_active":"2024-04-25T01:44:18.511855+0000","last_peered":"2024-04-25T01:44:18.511855+0000","last_clean":"2024-04-25T01:44:18.511855+0000","last_became_active":"2024-04-25T01:43:21.451178+0000","last_became_peered":"2024-04-25T01:43:21.451178+0000","last_unstale":"2024-04-25T01:44:18.511855+0000","last_undegraded":"2024-04-25T01:44:18.511855+0000","last_fullsized":"2024-04-25T01:44:18.511855+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:17.387498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.515741+0000","last_change":"2024-04-25T01:43:17.025580+0000","last_active":"2024-04-25T01:44:18.515741+0000","last_peered":"2024-04-25T01:44:18.515741+0000","last_clean":"2024-04-25T01:44:18.515741+0000","last_became_active":"2024-04-25T01:43:17.025267+0000","last_became_peered":"2024-04-25T01:43:17.025267+0000","last_unstale":"2024-04-25T01:44:18.515741+0000","last_undegraded":"2024-04-25T01:44:18.515741+0000","last_fullsized":"2024-04-25T01:44:18.515741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:04.922572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.924107+0000","last_change":"2024-04-25T01:44:18.517600+0000","last_active":"2024-04-25T01:44:46.924107+0000","last_peered":"2024-04-25T01:44:46.924107+0000","last_clean":"2024-04-25T01:44:46.924107+0000","last_became_active":"2024-04-25T01:44:18.517169+0000","last_became_peered":"2024-04-25T01:44:18.517169+0000","last_unstale":"2024-04-25T01:44:46.924107+0000","last_undegraded":"2024-04-25T01:44:46.924107+0000","last_fullsized":"2024-04-25T01:44:46.924107+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:11.389592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.513792+0000","last_change":"2024-04-25T01:43:21.437988+0000","last_active":"2024-04-25T01:44:18.513792+0000","last_peered":"2024-04-25T01:44:18.513792+0000","last_clean":"2024-04-25T01:44:18.513792+0000","last_became_active":"2024-04-25T01:43:21.437523+0000","last_became_peered":"2024-04-25T01:43:21.437523+0000","last_unstale":"2024-04-25T01:44:18.513792+0000","last_undegraded":"2024-04-25T01:44:18.513792+0000","last_fullsized":"2024-04-25T01:44:18.513792+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:20.415672+0000","last_clean_scrub_stamp":"2024-04-25T01:43:20.415672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:49:09.076123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:18.512093+0000","last_change":"2024-04-25T01:43:17.023058+0000","last_active":"2024-04-25T01:44:18.512093+0000","last_peered":"2024-04-25T01:44:18.512093+0000","last_clean":"2024-04-25T01:44:18.512093+0000","last_became_active":"2024-04-25T01:43:17.022730+0000","last_became_peered":"2024-04-25T01:43:17.022730+0000","last_unstale":"2024-04-25T01:44:18.512093+0000","last_undegraded":"2024-04-25T01:44:18.512093+0000","last_fullsized":"2024-04-25T01:44:18.512093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:15.996444+0000","last_clean_scrub_stamp":"2024-04-25T01:43:15.996444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:12.279161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.929742+0000","last_change":"2024-04-25T01:43:17.428832+0000","last_active":"2024-04-25T01:44:46.929742+0000","last_peered":"2024-04-25T01:44:46.929742+0000","last_clean":"2024-04-25T01:44:46.929742+0000","last_became_active":"2024-04-25T01:43:17.428493+0000","last_became_peered":"2024-04-25T01:43:17.428493+0000","last_unstale":"2024-04-25T01:44:46.929742+0000","last_undegraded":"2024-04-25T01:44:46.929742+0000","last_fullsized":"2024-04-25T01:44:46.929742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:47.224987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.923282+0000","last_change":"2024-04-25T01:43:17.429227+0000","last_active":"2024-04-25T01:44:46.923282+0000","last_peered":"2024-04-25T01:44:46.923282+0000","last_clean":"2024-04-25T01:44:46.923282+0000","last_became_active":"2024-04-25T01:43:17.428693+0000","last_became_peered":"2024-04-25T01:43:17.428693+0000","last_unstale":"2024-04-25T01:44:46.923282+0000","last_undegraded":"2024-04-25T01:44:46.923282+0000","last_fullsized":"2024-04-25T01:44:46.923282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:58.511414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.917515+0000","last_change":"2024-04-25T01:44:18.518461+0000","last_active":"2024-04-25T01:44:46.917515+0000","last_peered":"2024-04-25T01:44:46.917515+0000","last_clean":"2024-04-25T01:44:46.917515+0000","last_became_active":"2024-04-25T01:44:18.518110+0000","last_became_peered":"2024-04-25T01:44:18.518110+0000","last_unstale":"2024-04-25T01:44:46.917515+0000","last_undegraded":"2024-04-25T01:44:46.917515+0000","last_fullsized":"2024-04-25T01:44:46.917515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:58.551833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.920042+0000","last_change":"2024-04-25T01:43:17.421214+0000","last_active":"2024-04-25T01:44:46.920042+0000","last_peered":"2024-04-25T01:44:46.920042+0000","last_clean":"2024-04-25T01:44:46.920042+0000","last_became_active":"2024-04-25T01:43:17.420872+0000","last_became_peered":"2024-04-25T01:43:17.420872+0000","last_unstale":"2024-04-25T01:44:46.920042+0000","last_undegraded":"2024-04-25T01:44:46.920042+0000","last_fullsized":"2024-04-25T01:44:46.920042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:48.066191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.918399+0000","last_change":"2024-04-25T01:43:17.426011+0000","last_active":"2024-04-25T01:44:46.918399+0000","last_peered":"2024-04-25T01:44:46.918399+0000","last_clean":"2024-04-25T01:44:46.918399+0000","last_became_active":"2024-04-25T01:43:17.425629+0000","last_became_peered":"2024-04-25T01:43:17.425629+0000","last_unstale":"2024-04-25T01:44:46.918399+0000","last_undegraded":"2024-04-25T01:44:46.918399+0000","last_fullsized":"2024-04-25T01:44:46.918399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:12.606749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919975+0000","last_change":"2024-04-25T01:43:17.421374+0000","last_active":"2024-04-25T01:44:46.919975+0000","last_peered":"2024-04-25T01:44:46.919975+0000","last_clean":"2024-04-25T01:44:46.919975+0000","last_became_active":"2024-04-25T01:43:17.420864+0000","last_became_peered":"2024-04-25T01:43:17.420864+0000","last_unstale":"2024-04-25T01:44:46.919975+0000","last_undegraded":"2024-04-25T01:44:46.919975+0000","last_fullsized":"2024-04-25T01:44:46.919975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:39:17.517927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.922793+0000","last_change":"2024-04-25T01:43:17.431131+0000","last_active":"2024-04-25T01:44:46.922793+0000","last_peered":"2024-04-25T01:44:46.922793+0000","last_clean":"2024-04-25T01:44:46.922793+0000","last_became_active":"2024-04-25T01:43:17.430809+0000","last_became_peered":"2024-04-25T01:43:17.430809+0000","last_unstale":"2024-04-25T01:44:46.922793+0000","last_undegraded":"2024-04-25T01:44:46.922793+0000","last_fullsized":"2024-04-25T01:44:46.922793+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:48.261501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.930562+0000","last_change":"2024-04-25T01:43:17.419104+0000","last_active":"2024-04-25T01:44:46.930562+0000","last_peered":"2024-04-25T01:44:46.930562+0000","last_clean":"2024-04-25T01:44:46.930562+0000","last_became_active":"2024-04-25T01:43:17.418799+0000","last_became_peered":"2024-04-25T01:43:17.418799+0000","last_unstale":"2024-04-25T01:44:46.930562+0000","last_undegraded":"2024-04-25T01:44:46.930562+0000","last_fullsized":"2024-04-25T01:44:46.930562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:46.000847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.917714+0000","last_change":"2024-04-25T01:43:17.425801+0000","last_active":"2024-04-25T01:44:46.917714+0000","last_peered":"2024-04-25T01:44:46.917714+0000","last_clean":"2024-04-25T01:44:46.917714+0000","last_became_active":"2024-04-25T01:43:17.425133+0000","last_became_peered":"2024-04-25T01:43:17.425133+0000","last_unstale":"2024-04-25T01:44:46.917714+0000","last_undegraded":"2024-04-25T01:44:46.917714+0000","last_fullsized":"2024-04-25T01:44:46.917714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:54.229265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919025+0000","last_change":"2024-04-25T01:43:17.423931+0000","last_active":"2024-04-25T01:44:46.919025+0000","last_peered":"2024-04-25T01:44:46.919025+0000","last_clean":"2024-04-25T01:44:46.919025+0000","last_became_active":"2024-04-25T01:43:17.423477+0000","last_became_peered":"2024-04-25T01:43:17.423477+0000","last_unstale":"2024-04-25T01:44:46.919025+0000","last_undegraded":"2024-04-25T01:44:46.919025+0000","last_fullsized":"2024-04-25T01:44:46.919025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:31.538936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.919483+0000","last_change":"2024-04-25T01:43:17.428469+0000","last_active":"2024-04-25T01:44:46.919483+0000","last_peered":"2024-04-25T01:44:46.919483+0000","last_clean":"2024-04-25T01:44:46.919483+0000","last_became_active":"2024-04-25T01:43:17.428121+0000","last_became_peered":"2024-04-25T01:43:17.428121+0000","last_unstale":"2024-04-25T01:44:46.919483+0000","last_undegraded":"2024-04-25T01:44:46.919483+0000","last_fullsized":"2024-04-25T01:44:46.919483+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:27.563439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.937792+0000","last_change":"2024-04-25T01:43:17.433014+0000","last_active":"2024-04-25T01:44:46.937792+0000","last_peered":"2024-04-25T01:44:46.937792+0000","last_clean":"2024-04-25T01:44:46.937792+0000","last_became_active":"2024-04-25T01:43:17.432568+0000","last_became_peered":"2024-04-25T01:43:17.432568+0000","last_unstale":"2024-04-25T01:44:46.937792+0000","last_undegraded":"2024-04-25T01:44:46.937792+0000","last_fullsized":"2024-04-25T01:44:46.937792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:14:18.494922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.920697+0000","last_change":"2024-04-25T01:43:17.436079+0000","last_active":"2024-04-25T01:44:46.920697+0000","last_peered":"2024-04-25T01:44:46.920697+0000","last_clean":"2024-04-25T01:44:46.920697+0000","last_became_active":"2024-04-25T01:43:17.435735+0000","last_became_peered":"2024-04-25T01:43:17.435735+0000","last_unstale":"2024-04-25T01:44:46.920697+0000","last_undegraded":"2024-04-25T01:44:46.920697+0000","last_fullsized":"2024-04-25T01:44:46.920697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:59.593710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T01:44:46.924572+0000","last_change":"2024-04-25T01:43:17.424956+0000","last_active":"2024-04-25T01:44:46.924572+0000","last_peered":"2024-04-25T01:44:46.924572+0000","last_clean":"2024-04-25T01:44:46.924572+0000","last_became_active":"2024-04-25T01:43:17.424629+0000","last_became_peered":"2024-04-25T01:43:17.424629+0000","last_unstale":"2024-04-25T01:44:46.924572+0000","last_undegraded":"2024-04-25T01:44:46.924572+0000","last_fullsized":"2024-04-25T01:44:46.924572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:43:17.005628+0000","last_clean_scrub_stamp":"2024-04-25T01:43:17.005628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:58.006952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245924,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71623,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":750,"num_read_kb":455,"num_write":253,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":214869,"data_compressed":4824,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351009,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":506404,"data_compressed":20650,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612646,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":1160921,"data_compressed":58104,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":509898,"data_compressed":20650,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037809,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":506180,"data_compressed":20650,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299445,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":1103350,"data_compressed":56496,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":501306,"data_compressed":20650,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":565941,"data_compressed":22258,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":503294,"data_compressed":20650,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":567225,"data_compressed":22258,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":501014,"data_compressed":20650,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":505912,"data_compressed":20650,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":1094304,"data_compressed":56496,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34180,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35846,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65357,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66641,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65873,"data_compressed":1608,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:44:59.511 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T01:44:59.511 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T01:44:59.511 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T01:44:59.512 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T01:44:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:45:00.138 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:59 smithi078 ceph-mon[29469]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:45:00.138 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:44:59 smithi078 ceph-mon[29469]: pgmap v343: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-25T01:45:00.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:59 smithi152 ceph-mon[32465]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:45:00.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:44:59 smithi152 ceph-mon[32465]: pgmap v343: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-25T01:45:00.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:59 smithi196 ceph-mon[32978]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:45:00.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:44:59 smithi196 ceph-mon[32978]: pgmap v343: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-25T01:45:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.515+0000 7fe1c0868700 1 -- 172.21.15.78:0/2249411040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1bc1064b0 msgr2=0x7fe1bc10e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.515+0000 7fe1c0868700 1 --2- 172.21.15.78:0/2249411040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1bc1064b0 0x7fe1bc10e0c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0099c0 tx=0x7fe1ac009cd0 comp rx=0 tx=0).stop 2024-04-25T01:45:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.515+0000 7fe1c0868700 1 -- 172.21.15.78:0/2249411040 shutdown_connections 2024-04-25T01:45:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.515+0000 7fe1c0868700 1 --2- 172.21.15.78:0/2249411040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc101b90 0x7fe1bc101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.515+0000 7fe1c0868700 1 --2- 172.21.15.78:0/2249411040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1bc1064b0 0x7fe1bc10e0c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.515+0000 7fe1c0868700 1 --2- 172.21.15.78:0/2249411040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1bc102540 0x7fe1bc105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.515+0000 7fe1c0868700 1 -- 172.21.15.78:0/2249411040 >> 172.21.15.78:0/2249411040 conn(0x7fe1bc0777a0 msgr2=0x7fe1bc0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.515+0000 7fe1c0868700 1 -- 172.21.15.78:0/2249411040 shutdown_connections 2024-04-25T01:45:01.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.515+0000 7fe1c0868700 1 -- 172.21.15.78:0/2249411040 wait complete. 2024-04-25T01:45:01.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.516+0000 7fe1c0868700 1 Processor -- start 2024-04-25T01:45:01.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.516+0000 7fe1c0868700 1 -- start start 2024-04-25T01:45:01.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.516+0000 7fe1c0868700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1bc101b90 0x7fe1bc19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.516+0000 7fe1c0868700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1bc102540 0x7fe1bc19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.516+0000 7fe1c0868700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc1064b0 0x7fe1bc198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.516+0000 7fe1c0868700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc1114f0 con 0x7fe1bc102540 2024-04-25T01:45:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.516+0000 7fe1c0868700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc111370 con 0x7fe1bc101b90 2024-04-25T01:45:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.516+0000 7fe1c0868700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc111670 con 0x7fe1bc1064b0 2024-04-25T01:45:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.516+0000 7fe1b9d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1bc102540 0x7fe1bc19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.516+0000 7fe1b9d9b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1bc102540 0x7fe1bc19ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40792/0 (socket says 172.21.15.78:40792) 2024-04-25T01:45:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.516+0000 7fe1b9d9b700 1 -- 172.21.15.78:0/155073135 learned_addr learned my addr 172.21.15.78:0/155073135 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.516+0000 7fe1ba59c700 1 --2- 172.21.15.78:0/155073135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1bc101b90 0x7fe1bc19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.516+0000 7fe1bad9d700 1 --2- 172.21.15.78:0/155073135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc1064b0 0x7fe1bc198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.517+0000 7fe1b9d9b700 1 -- 172.21.15.78:0/155073135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc1064b0 msgr2=0x7fe1bc198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.517+0000 7fe1b9d9b700 1 --2- 172.21.15.78:0/155073135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc1064b0 0x7fe1bc198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:01.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.517+0000 7fe1b9d9b700 1 -- 172.21.15.78:0/155073135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1bc101b90 msgr2=0x7fe1bc19e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.517+0000 7fe1b9d9b700 1 --2- 172.21.15.78:0/155073135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1bc101b90 0x7fe1bc19e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.517+0000 7fe1b9d9b700 1 -- 172.21.15.78:0/155073135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc198fd0 con 0x7fe1bc102540 2024-04-25T01:45:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.517+0000 7fe1b9d9b700 1 --2- 172.21.15.78:0/155073135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1bc102540 0x7fe1bc19ebf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c8f0 tx=0x7fe1b000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.517+0000 7fe1ab7fe700 1 -- 172.21.15.78:0/155073135 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b00078d0 con 0x7fe1bc102540 2024-04-25T01:45:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.517+0000 7fe1c0868700 1 -- 172.21.15.78:0/155073135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1bc1992c0 con 0x7fe1bc102540 2024-04-25T01:45:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.517+0000 7fe1ab7fe700 1 -- 172.21.15.78:0/155073135 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b000f450 con 0x7fe1bc102540 2024-04-25T01:45:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.517+0000 7fe1ab7fe700 1 -- 172.21.15.78:0/155073135 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0018640 con 0x7fe1bc102540 2024-04-25T01:45:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.517+0000 7fe1c0868700 1 -- 172.21.15.78:0/155073135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1bc1a5580 con 0x7fe1bc102540 2024-04-25T01:45:01.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.518+0000 7fe1c0868700 1 -- 172.21.15.78:0/155073135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1bc102940 con 0x7fe1bc102540 2024-04-25T01:45:01.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.519+0000 7fe1ab7fe700 1 -- 172.21.15.78:0/155073135 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe1b000f600 con 0x7fe1bc102540 2024-04-25T01:45:01.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.520+0000 7fe1ab7fe700 1 --2- 172.21.15.78:0/155073135 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe1a40a2c20 0x7fe1a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:01.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.520+0000 7fe1ba59c700 1 --2- 172.21.15.78:0/155073135 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe1a40a2c20 0x7fe1a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:01.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.520+0000 7fe1ab7fe700 1 -- 172.21.15.78:0/155073135 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fe1b00cf250 con 0x7fe1bc102540 2024-04-25T01:45:01.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.520+0000 7fe1ba59c700 1 --2- 172.21.15.78:0/155073135 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe1a40a2c20 0x7fe1a40a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40060b0 tx=0x7fe1b400c040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:01.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.524+0000 7fe1ab7fe700 1 -- 172.21.15.78:0/155073135 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe1b0014030 con 0x7fe1bc102540 2024-04-25T01:45:01.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.764+0000 7fe1c0868700 1 -- 172.21.15.78:0/155073135 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe1bc199da0 con 0x7fe1bc102540 2024-04-25T01:45:01.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.764+0000 7fe1ab7fe700 1 -- 172.21.15.78:0/155073135 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe1b009bea0 con 0x7fe1bc102540 2024-04-25T01:45:01.757 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:01.758 INFO:teuthology.orchestra.run.smithi078.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T01:45:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.767+0000 7fe1c0868700 1 -- 172.21.15.78:0/155073135 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe1a40a2c20 msgr2=0x7fe1a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.767+0000 7fe1c0868700 1 --2- 172.21.15.78:0/155073135 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe1a40a2c20 0x7fe1a40a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40060b0 tx=0x7fe1b400c040 comp rx=0 tx=0).stop 2024-04-25T01:45:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.767+0000 7fe1c0868700 1 -- 172.21.15.78:0/155073135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1bc102540 msgr2=0x7fe1bc19ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.767+0000 7fe1c0868700 1 --2- 172.21.15.78:0/155073135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1bc102540 0x7fe1bc19ebf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c8f0 tx=0x7fe1b000cc00 comp rx=0 tx=0).stop 2024-04-25T01:45:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.767+0000 7fe1c0868700 1 -- 172.21.15.78:0/155073135 shutdown_connections 2024-04-25T01:45:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.767+0000 7fe1c0868700 1 --2- 172.21.15.78:0/155073135 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe1a40a2c20 0x7fe1a40a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.767+0000 7fe1c0868700 1 --2- 172.21.15.78:0/155073135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1bc1064b0 0x7fe1bc198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.767+0000 7fe1c0868700 1 --2- 172.21.15.78:0/155073135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1bc101b90 0x7fe1bc19e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.767+0000 7fe1c0868700 1 --2- 172.21.15.78:0/155073135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1bc102540 0x7fe1bc19ebf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.767+0000 7fe1c0868700 1 -- 172.21.15.78:0/155073135 >> 172.21.15.78:0/155073135 conn(0x7fe1bc0777a0 msgr2=0x7fe1bc100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.767+0000 7fe1c0868700 1 -- 172.21.15.78:0/155073135 shutdown_connections 2024-04-25T01:45:01.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:01.767+0000 7fe1c0868700 1 -- 172.21.15.78:0/155073135 wait complete. 2024-04-25T01:45:02.168 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:01 smithi078 ceph-mon[29469]: pgmap v344: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-25T01:45:02.168 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:01 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/155073135' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T01:45:02.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:01 smithi152 ceph-mon[32465]: pgmap v344: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-25T01:45:02.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:01 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/155073135' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T01:45:02.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:01 smithi196 ceph-mon[32978]: pgmap v344: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-25T01:45:02.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:01 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/155073135' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T01:45:02.564 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T01:45:02.565 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:45:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.675+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/66824617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1c4102230 msgr2=0x7ff1c4108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.675+0000 7ff1ca5a2640 1 --2- 172.21.15.78:0/66824617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1c4102230 0x7ff1c4108ac0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009ca0 tx=0x7ff1bc02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.676+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/66824617 shutdown_connections 2024-04-25T01:45:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.676+0000 7ff1ca5a2640 1 --2- 172.21.15.78:0/66824617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1c4102230 0x7ff1c4108ac0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.676+0000 7ff1ca5a2640 1 --2- 172.21.15.78:0/66824617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1c4101870 0x7ff1c4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.676+0000 7ff1ca5a2640 1 --2- 172.21.15.78:0/66824617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1c4100680 0x7ff1c4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.676+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/66824617 >> 172.21.15.78:0/66824617 conn(0x7ff1c40fbbf0 msgr2=0x7ff1c40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.676+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/66824617 shutdown_connections 2024-04-25T01:45:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.676+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/66824617 wait complete. 2024-04-25T01:45:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.676+0000 7ff1ca5a2640 1 Processor -- start 2024-04-25T01:45:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.676+0000 7ff1ca5a2640 1 -- start start 2024-04-25T01:45:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.676+0000 7ff1ca5a2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1c4100680 0x7ff1c4197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.676+0000 7ff1ca5a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1c4101870 0x7ff1c4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1ca5a2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1c4102230 0x7ff1c419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1ca5a2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c410c3f0 con 0x7ff1c4101870 2024-04-25T01:45:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1ca5a2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c410c270 con 0x7ff1c4102230 2024-04-25T01:45:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1ca5a2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c410c570 con 0x7ff1c4100680 2024-04-25T01:45:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1c4101870 0x7ff1c4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1c4101870 0x7ff1c4198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40810/0 (socket says 172.21.15.78:40810) 2024-04-25T01:45:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1c37fe640 1 -- 172.21.15.78:0/3083733924 learned_addr learned my addr 172.21.15.78:0/3083733924 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1c3fff640 1 --2- 172.21.15.78:0/3083733924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1c4100680 0x7ff1c4197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1c8b3f640 1 --2- 172.21.15.78:0/3083733924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1c4102230 0x7ff1c419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1c37fe640 1 -- 172.21.15.78:0/3083733924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1c4100680 msgr2=0x7ff1c4197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1c37fe640 1 --2- 172.21.15.78:0/3083733924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1c4100680 0x7ff1c4197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1c37fe640 1 -- 172.21.15.78:0/3083733924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1c4102230 msgr2=0x7ff1c419f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1c37fe640 1 --2- 172.21.15.78:0/3083733924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1c4102230 0x7ff1c419f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1c37fe640 1 -- 172.21.15.78:0/3083733924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c419f7f0 con 0x7ff1c4101870 2024-04-25T01:45:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1c37fe640 1 --2- 172.21.15.78:0/3083733924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1c4101870 0x7ff1c4198110 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800b4a0 tx=0x7ff1b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1c17fa640 1 -- 172.21.15.78:0/3083733924 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8014070 con 0x7ff1c4101870 2024-04-25T01:45:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/3083733924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c419fae0 con 0x7ff1c4101870 2024-04-25T01:45:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1c17fa640 1 -- 172.21.15.78:0/3083733924 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1b8010af0 con 0x7ff1c4101870 2024-04-25T01:45:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1c17fa640 1 -- 172.21.15.78:0/3083733924 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8017d10 con 0x7ff1c4101870 2024-04-25T01:45:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.677+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/3083733924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c41a0030 con 0x7ff1c4101870 2024-04-25T01:45:02.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.678+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/3083733924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c41a02c0 con 0x7ff1c4101870 2024-04-25T01:45:02.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.679+0000 7ff1c17fa640 1 -- 172.21.15.78:0/3083733924 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff1b8017070 con 0x7ff1c4101870 2024-04-25T01:45:02.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.680+0000 7ff1c17fa640 1 --2- 172.21.15.78:0/3083733924 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff1ac0a2c10 0x7ff1ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:02.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.680+0000 7ff1c3fff640 1 --2- 172.21.15.78:0/3083733924 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff1ac0a2c10 0x7ff1ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:02.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.680+0000 7ff1c17fa640 1 -- 172.21.15.78:0/3083733924 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7ff1b80ceb40 con 0x7ff1c4101870 2024-04-25T01:45:02.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.680+0000 7ff1c3fff640 1 --2- 172.21.15.78:0/3083733924 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff1ac0a2c10 0x7ff1ac0a50d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff1c41016d0 tx=0x7ff1b4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:02.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.684+0000 7ff1c17fa640 1 -- 172.21.15.78:0/3083733924 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff1b809b760 con 0x7ff1c4101870 2024-04-25T01:45:02.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.861+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/3083733924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1c41a0490 con 0x7ff1c4101870 2024-04-25T01:45:02.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.862+0000 7ff1c17fa640 1 -- 172.21.15.78:0/3083733924 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24592 (secure 0 0 0) 0x7ff1b809b060 con 0x7ff1c4101870 2024-04-25T01:45:02.855 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:02.855 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:44:18.500577+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:45:02.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.864+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/3083733924 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff1ac0a2c10 msgr2=0x7ff1ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:02.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.864+0000 7ff1ca5a2640 1 --2- 172.21.15.78:0/3083733924 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff1ac0a2c10 0x7ff1ac0a50d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff1c41016d0 tx=0x7ff1b4008040 comp rx=0 tx=0).stop 2024-04-25T01:45:02.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.864+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/3083733924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1c4101870 msgr2=0x7ff1c4198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:02.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.864+0000 7ff1ca5a2640 1 --2- 172.21.15.78:0/3083733924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1c4101870 0x7ff1c4198110 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800b4a0 tx=0x7ff1b800b960 comp rx=0 tx=0).stop 2024-04-25T01:45:02.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.864+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/3083733924 shutdown_connections 2024-04-25T01:45:02.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.864+0000 7ff1ca5a2640 1 --2- 172.21.15.78:0/3083733924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1c4102230 0x7ff1c419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.864+0000 7ff1ca5a2640 1 --2- 172.21.15.78:0/3083733924 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff1ac0a2c10 0x7ff1ac0a50d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.864+0000 7ff1ca5a2640 1 --2- 172.21.15.78:0/3083733924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1c4101870 0x7ff1c4198110 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.864+0000 7ff1ca5a2640 1 --2- 172.21.15.78:0/3083733924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1c4100680 0x7ff1c4197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.864+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/3083733924 >> 172.21.15.78:0/3083733924 conn(0x7ff1c40fbbf0 msgr2=0x7ff1c40fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:02.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.864+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/3083733924 shutdown_connections 2024-04-25T01:45:02.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.864+0000 7ff1ca5a2640 1 -- 172.21.15.78:0/3083733924 wait complete. 2024-04-25T01:45:02.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:38:39.136418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:43:15.613881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:43:16.600270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:43:19.650457+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:45:02.870 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:45:02.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 -- 172.21.15.78:0/3036872814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03181017a0 msgr2=0x7f0318101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:02.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 --2- 172.21.15.78:0/3036872814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03181017a0 0x7f0318101bc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0310009900 tx=0x7f031002a410 comp rx=0 tx=0).stop 2024-04-25T01:45:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 -- 172.21.15.78:0/3036872814 shutdown_connections 2024-04-25T01:45:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 --2- 172.21.15.78:0/3036872814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0318103350 0x7f0318109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 --2- 172.21.15.78:0/3036872814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0318102990 0x7f0318102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 --2- 172.21.15.78:0/3036872814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03181017a0 0x7f0318101bc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 -- 172.21.15.78:0/3036872814 >> 172.21.15.78:0/3036872814 conn(0x7f03180fcd10 msgr2=0x7f03180ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 -- 172.21.15.78:0/3036872814 shutdown_connections 2024-04-25T01:45:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 -- 172.21.15.78:0/3036872814 wait complete. 2024-04-25T01:45:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 Processor -- start 2024-04-25T01:45:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 -- start start 2024-04-25T01:45:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03181017a0 0x7f0318197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0318102990 0x7f0318198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0318103350 0x7f031819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f031810d410 con 0x7f03181017a0 2024-04-25T01:45:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f031810d290 con 0x7f0318103350 2024-04-25T01:45:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f03200a7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f031810d590 con 0x7f0318102990 2024-04-25T01:45:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f031de43640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03181017a0 0x7f0318197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.979+0000 7f031de43640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03181017a0 0x7f0318197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40828/0 (socket says 172.21.15.78:40828) 2024-04-25T01:45:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f031de43640 1 -- 172.21.15.78:0/3862399580 learned_addr learned my addr 172.21.15.78:0/3862399580 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f031d642640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0318102990 0x7f0318198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f031e644640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0318103350 0x7f031819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f031de43640 1 -- 172.21.15.78:0/3862399580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0318102990 msgr2=0x7f0318198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f031de43640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0318102990 0x7f0318198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f031de43640 1 -- 172.21.15.78:0/3862399580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0318103350 msgr2=0x7f031819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f031de43640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0318103350 0x7f031819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f031de43640 1 -- 172.21.15.78:0/3862399580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031819f700 con 0x7f03181017a0 2024-04-25T01:45:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f031e644640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0318103350 0x7f031819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:02.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f031de43640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03181017a0 0x7f0318197ae0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0310031610 tx=0x7f03100043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f0316ffd640 1 -- 172.21.15.78:0/3862399580 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031003f030 con 0x7f03181017a0 2024-04-25T01:45:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f0316ffd640 1 -- 172.21.15.78:0/3862399580 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0310003ce0 con 0x7f03181017a0 2024-04-25T01:45:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f03200a7640 1 -- 172.21.15.78:0/3862399580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031819f990 con 0x7f03181017a0 2024-04-25T01:45:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f0316ffd640 1 -- 172.21.15.78:0/3862399580 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310035070 con 0x7f03181017a0 2024-04-25T01:45:02.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.980+0000 7f03200a7640 1 -- 172.21.15.78:0/3862399580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f031819fe80 con 0x7f03181017a0 2024-04-25T01:45:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.981+0000 7f03200a7640 1 -- 172.21.15.78:0/3862399580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0318108b60 con 0x7f03181017a0 2024-04-25T01:45:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.982+0000 7f0316ffd640 1 -- 172.21.15.78:0/3862399580 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0310042030 con 0x7f03181017a0 2024-04-25T01:45:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.983+0000 7f0316ffd640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f03000a2b90 0x7f03000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.983+0000 7f031d642640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f03000a2b90 0x7f03000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.983+0000 7f0316ffd640 1 -- 172.21.15.78:0/3862399580 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f03100ef950 con 0x7f03181017a0 2024-04-25T01:45:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.983+0000 7f031d642640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f03000a2b90 0x7f03000a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0318198ff0 tx=0x7f030c00a250 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:02.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:02.987+0000 7f0316ffd640 1 -- 172.21.15.78:0/3862399580 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f03100bc570 con 0x7f03181017a0 2024-04-25T01:45:03.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.163+0000 7f03200a7640 1 -- 172.21.15.78:0/3862399580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0318062340 con 0x7f03181017a0 2024-04-25T01:45:03.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.164+0000 7f0316ffd640 1 -- 172.21.15.78:0/3862399580 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f03100bbe70 con 0x7f03181017a0 2024-04-25T01:45:03.157 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T01:45:03.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.166+0000 7f03200a7640 1 -- 172.21.15.78:0/3862399580 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f03000a2b90 msgr2=0x7f03000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.166+0000 7f03200a7640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f03000a2b90 0x7f03000a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0318198ff0 tx=0x7f030c00a250 comp rx=0 tx=0).stop 2024-04-25T01:45:03.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.166+0000 7f03200a7640 1 -- 172.21.15.78:0/3862399580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03181017a0 msgr2=0x7f0318197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.166+0000 7f03200a7640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03181017a0 0x7f0318197ae0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0310031610 tx=0x7f03100043b0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.166+0000 7f03200a7640 1 -- 172.21.15.78:0/3862399580 shutdown_connections 2024-04-25T01:45:03.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.166+0000 7f03200a7640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0318103350 0x7f031819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.166+0000 7f03200a7640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0318102990 0x7f0318198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.166+0000 7f03200a7640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f03000a2b90 0x7f03000a5050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.166+0000 7f03200a7640 1 --2- 172.21.15.78:0/3862399580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03181017a0 0x7f0318197ae0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.166+0000 7f03200a7640 1 -- 172.21.15.78:0/3862399580 >> 172.21.15.78:0/3862399580 conn(0x7f03180fcd10 msgr2=0x7f0318105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:03.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.166+0000 7f03200a7640 1 -- 172.21.15.78:0/3862399580 shutdown_connections 2024-04-25T01:45:03.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.166+0000 7f03200a7640 1 -- 172.21.15.78:0/3862399580 wait complete. 2024-04-25T01:45:03.169 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:02 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3083733924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:03.170 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:45:03.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.280+0000 7f21467c8640 1 -- 172.21.15.78:0/2543585689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2140100ff0 msgr2=0x7f2140101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.280+0000 7f21467c8640 1 --2- 172.21.15.78:0/2543585689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2140100ff0 0x7f2140101470 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2134009900 tx=0x7f213402a410 comp rx=0 tx=0).stop 2024-04-25T01:45:03.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.280+0000 7f21467c8640 1 -- 172.21.15.78:0/2543585689 shutdown_connections 2024-04-25T01:45:03.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.280+0000 7f21467c8640 1 --2- 172.21.15.78:0/2543585689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21401019b0 0x7f2140108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.280+0000 7f21467c8640 1 --2- 172.21.15.78:0/2543585689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2140100ff0 0x7f2140101470 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.280+0000 7f21467c8640 1 --2- 172.21.15.78:0/2543585689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21400ffe00 0x7f2140100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.280+0000 7f21467c8640 1 -- 172.21.15.78:0/2543585689 >> 172.21.15.78:0/2543585689 conn(0x7f21400fb370 msgr2=0x7f21400fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:03.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.280+0000 7f21467c8640 1 -- 172.21.15.78:0/2543585689 shutdown_connections 2024-04-25T01:45:03.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.280+0000 7f21467c8640 1 -- 172.21.15.78:0/2543585689 wait complete. 2024-04-25T01:45:03.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.281+0000 7f21467c8640 1 Processor -- start 2024-04-25T01:45:03.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.281+0000 7f21467c8640 1 -- start start 2024-04-25T01:45:03.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.281+0000 7f21467c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21400ffe00 0x7f21401969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:03.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.281+0000 7f21467c8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2140100ff0 0x7f2140196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:03.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.281+0000 7f21467c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21401019b0 0x7f214019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:03.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.281+0000 7f21467c8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f214010c3b0 con 0x7f21400ffe00 2024-04-25T01:45:03.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.281+0000 7f21467c8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f214010c230 con 0x7f2140100ff0 2024-04-25T01:45:03.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.281+0000 7f21467c8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f214010c530 con 0x7f21401019b0 2024-04-25T01:45:03.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.281+0000 7f213ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21400ffe00 0x7f21401969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:03.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.281+0000 7f213ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21400ffe00 0x7f21401969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40836/0 (socket says 172.21.15.78:40836) 2024-04-25T01:45:03.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.281+0000 7f213ffff640 1 -- 172.21.15.78:0/1719500078 learned_addr learned my addr 172.21.15.78:0/1719500078 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:03.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.281+0000 7f213f7fe640 1 --2- 172.21.15.78:0/1719500078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2140100ff0 0x7f2140196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:03.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.281+0000 7f2144d65640 1 --2- 172.21.15.78:0/1719500078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21401019b0 0x7f214019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:03.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.282+0000 7f213ffff640 1 -- 172.21.15.78:0/1719500078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21401019b0 msgr2=0x7f214019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.282+0000 7f213ffff640 1 --2- 172.21.15.78:0/1719500078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21401019b0 0x7f214019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.282+0000 7f213ffff640 1 -- 172.21.15.78:0/1719500078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2140100ff0 msgr2=0x7f2140196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.282+0000 7f213ffff640 1 --2- 172.21.15.78:0/1719500078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2140100ff0 0x7f2140196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.282+0000 7f213ffff640 1 -- 172.21.15.78:0/1719500078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214019e610 con 0x7f21400ffe00 2024-04-25T01:45:03.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.282+0000 7f213ffff640 1 --2- 172.21.15.78:0/1719500078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21400ffe00 0x7f21401969f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f213000c970 tx=0x7f213000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:03.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.282+0000 7f213d7fa640 1 -- 172.21.15.78:0/1719500078 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130004220 con 0x7f21400ffe00 2024-04-25T01:45:03.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.282+0000 7f213d7fa640 1 -- 172.21.15.78:0/1719500078 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21300043c0 con 0x7f21400ffe00 2024-04-25T01:45:03.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.282+0000 7f21467c8640 1 -- 172.21.15.78:0/1719500078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214019e900 con 0x7f21400ffe00 2024-04-25T01:45:03.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.282+0000 7f213d7fa640 1 -- 172.21.15.78:0/1719500078 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213000f450 con 0x7f21400ffe00 2024-04-25T01:45:03.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.282+0000 7f21467c8640 1 -- 172.21.15.78:0/1719500078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214019eea0 con 0x7f21400ffe00 2024-04-25T01:45:03.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.283+0000 7f21467c8640 1 -- 172.21.15.78:0/1719500078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2140107120 con 0x7f21400ffe00 2024-04-25T01:45:03.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.284+0000 7f213d7fa640 1 -- 172.21.15.78:0/1719500078 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f21300079b0 con 0x7f21400ffe00 2024-04-25T01:45:03.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.285+0000 7f213d7fa640 1 --2- 172.21.15.78:0/1719500078 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f21280a2b90 0x7f21280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:03.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.285+0000 7f213f7fe640 1 --2- 172.21.15.78:0/1719500078 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f21280a2b90 0x7f21280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:03.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.285+0000 7f213d7fa640 1 -- 172.21.15.78:0/1719500078 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f21300cf110 con 0x7f21400ffe00 2024-04-25T01:45:03.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.285+0000 7f213f7fe640 1 --2- 172.21.15.78:0/1719500078 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f21280a2b90 0x7f21280a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2134031440 tx=0x7f21340315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:03.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.289+0000 7f213d7fa640 1 -- 172.21.15.78:0/1719500078 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2130014030 con 0x7f21400ffe00 2024-04-25T01:45:03.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:02 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3083733924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:03.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:02 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3083733924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:03.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.467+0000 7f21467c8640 1 -- 172.21.15.78:0/1719500078 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2140062340 con 0x7f21400ffe00 2024-04-25T01:45:03.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.467+0000 7f213d7fa640 1 -- 172.21.15.78:0/1719500078 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f213009bd30 con 0x7f21400ffe00 2024-04-25T01:45:03.460 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:45:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.469+0000 7f21467c8640 1 -- 172.21.15.78:0/1719500078 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f21280a2b90 msgr2=0x7f21280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.469+0000 7f21467c8640 1 --2- 172.21.15.78:0/1719500078 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f21280a2b90 0x7f21280a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2134031440 tx=0x7f21340315b0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.469+0000 7f21467c8640 1 -- 172.21.15.78:0/1719500078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21400ffe00 msgr2=0x7f21401969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.469+0000 7f21467c8640 1 --2- 172.21.15.78:0/1719500078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21400ffe00 0x7f21401969f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f213000c970 tx=0x7f213000ce30 comp rx=0 tx=0).stop 2024-04-25T01:45:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.470+0000 7f21467c8640 1 -- 172.21.15.78:0/1719500078 shutdown_connections 2024-04-25T01:45:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.470+0000 7f21467c8640 1 --2- 172.21.15.78:0/1719500078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21401019b0 0x7f214019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.470+0000 7f21467c8640 1 --2- 172.21.15.78:0/1719500078 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2140100ff0 0x7f2140196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.470+0000 7f21467c8640 1 --2- 172.21.15.78:0/1719500078 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f21280a2b90 0x7f21280a5050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.470+0000 7f21467c8640 1 --2- 172.21.15.78:0/1719500078 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21400ffe00 0x7f21401969f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.470+0000 7f21467c8640 1 -- 172.21.15.78:0/1719500078 >> 172.21.15.78:0/1719500078 conn(0x7f21400fb370 msgr2=0x7f21401042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.470+0000 7f21467c8640 1 -- 172.21.15.78:0/1719500078 shutdown_connections 2024-04-25T01:45:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.470+0000 7f21467c8640 1 -- 172.21.15.78:0/1719500078 wait complete. 2024-04-25T01:45:03.474 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:45:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.582+0000 7f475e627640 1 -- 172.21.15.78:0/265960224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47581006a0 msgr2=0x7f4758100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.582+0000 7f475e627640 1 --2- 172.21.15.78:0/265960224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47581006a0 0x7f4758100ac0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4748009900 tx=0x7f474802a410 comp rx=0 tx=0).stop 2024-04-25T01:45:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.582+0000 7f475e627640 1 -- 172.21.15.78:0/265960224 shutdown_connections 2024-04-25T01:45:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.582+0000 7f475e627640 1 --2- 172.21.15.78:0/265960224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4758102250 0x7f4758108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.582+0000 7f475e627640 1 --2- 172.21.15.78:0/265960224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4758101890 0x7f4758101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.582+0000 7f475e627640 1 --2- 172.21.15.78:0/265960224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47581006a0 0x7f4758100ac0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.582+0000 7f475e627640 1 -- 172.21.15.78:0/265960224 >> 172.21.15.78:0/265960224 conn(0x7f47580fbc10 msgr2=0x7f47580fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.582+0000 7f475e627640 1 -- 172.21.15.78:0/265960224 shutdown_connections 2024-04-25T01:45:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.582+0000 7f475e627640 1 -- 172.21.15.78:0/265960224 wait complete. 2024-04-25T01:45:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475e627640 1 Processor -- start 2024-04-25T01:45:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475e627640 1 -- start start 2024-04-25T01:45:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475e627640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47581006a0 0x7f475806a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475e627640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4758101890 0x7f475819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475e627640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4758102250 0x7f475819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475e627640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f475810c320 con 0x7f4758102250 2024-04-25T01:45:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475e627640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f475810c1a0 con 0x7f4758101890 2024-04-25T01:45:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475e627640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f475810c4a0 con 0x7f47581006a0 2024-04-25T01:45:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475cbc4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4758102250 0x7f475819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475cbc4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4758102250 0x7f475819ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40844/0 (socket says 172.21.15.78:40844) 2024-04-25T01:45:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475cbc4640 1 -- 172.21.15.78:0/361010690 learned_addr learned my addr 172.21.15.78:0/361010690 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:03.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f4757fff640 1 --2- 172.21.15.78:0/361010690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47581006a0 0x7f475806a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f47577fe640 1 --2- 172.21.15.78:0/361010690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4758101890 0x7f475819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475cbc4640 1 -- 172.21.15.78:0/361010690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47581006a0 msgr2=0x7f475806a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475cbc4640 1 --2- 172.21.15.78:0/361010690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47581006a0 0x7f475806a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475cbc4640 1 -- 172.21.15.78:0/361010690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4758101890 msgr2=0x7f475819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475cbc4640 1 --2- 172.21.15.78:0/361010690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4758101890 0x7f475819c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.583+0000 7f475cbc4640 1 -- 172.21.15.78:0/361010690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475819f780 con 0x7f4758102250 2024-04-25T01:45:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.584+0000 7f475cbc4640 1 --2- 172.21.15.78:0/361010690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4758102250 0x7f475819ee70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f475000eaa0 tx=0x7f475000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.584+0000 7f47557fa640 1 -- 172.21.15.78:0/361010690 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475000cdb0 con 0x7f4758102250 2024-04-25T01:45:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.584+0000 7f47557fa640 1 -- 172.21.15.78:0/361010690 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4750005020 con 0x7f4758102250 2024-04-25T01:45:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.584+0000 7f475e627640 1 -- 172.21.15.78:0/361010690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475819fa10 con 0x7f4758102250 2024-04-25T01:45:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.584+0000 7f47557fa640 1 -- 172.21.15.78:0/361010690 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47500116b0 con 0x7f4758102250 2024-04-25T01:45:03.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.584+0000 7f475e627640 1 -- 172.21.15.78:0/361010690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f475819ff60 con 0x7f4758102250 2024-04-25T01:45:03.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.585+0000 7f475e627640 1 -- 172.21.15.78:0/361010690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47581079e0 con 0x7f4758102250 2024-04-25T01:45:03.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.586+0000 7f47557fa640 1 -- 172.21.15.78:0/361010690 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4750005560 con 0x7f4758102250 2024-04-25T01:45:03.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.586+0000 7f47557fa640 1 --2- 172.21.15.78:0/361010690 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f47400a2b90 0x7f47400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:03.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.586+0000 7f4757fff640 1 --2- 172.21.15.78:0/361010690 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f47400a2b90 0x7f47400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:03.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.586+0000 7f47557fa640 1 -- 172.21.15.78:0/361010690 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f47500cf5f0 con 0x7f4758102250 2024-04-25T01:45:03.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.587+0000 7f4757fff640 1 --2- 172.21.15.78:0/361010690 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f47400a2b90 0x7f47400a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4748005ab0 tx=0x7f4748000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:03.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.591+0000 7f47557fa640 1 -- 172.21.15.78:0/361010690 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4750016030 con 0x7f4758102250 2024-04-25T01:45:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.768+0000 7f475e627640 1 -- 172.21.15.78:0/361010690 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4758062340 con 0x7f4758102250 2024-04-25T01:45:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.769+0000 7f47557fa640 1 -- 172.21.15.78:0/361010690 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f475009c210 con 0x7f4758102250 2024-04-25T01:45:03.762 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T01:45:03.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.771+0000 7f475e627640 1 -- 172.21.15.78:0/361010690 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f47400a2b90 msgr2=0x7f47400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.771+0000 7f475e627640 1 --2- 172.21.15.78:0/361010690 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f47400a2b90 0x7f47400a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4748005ab0 tx=0x7f4748000bc0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.771+0000 7f475e627640 1 -- 172.21.15.78:0/361010690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4758102250 msgr2=0x7f475819ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.771+0000 7f475e627640 1 --2- 172.21.15.78:0/361010690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4758102250 0x7f475819ee70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f475000eaa0 tx=0x7f475000ef60 comp rx=0 tx=0).stop 2024-04-25T01:45:03.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.771+0000 7f475e627640 1 -- 172.21.15.78:0/361010690 shutdown_connections 2024-04-25T01:45:03.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.771+0000 7f475e627640 1 --2- 172.21.15.78:0/361010690 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f47400a2b90 0x7f47400a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.771+0000 7f475e627640 1 --2- 172.21.15.78:0/361010690 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4758102250 0x7f475819ee70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.771+0000 7f475e627640 1 --2- 172.21.15.78:0/361010690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4758101890 0x7f475819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.771+0000 7f475e627640 1 --2- 172.21.15.78:0/361010690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47581006a0 0x7f475806a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.771+0000 7f475e627640 1 -- 172.21.15.78:0/361010690 >> 172.21.15.78:0/361010690 conn(0x7f47580fbc10 msgr2=0x7f4758104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.771+0000 7f475e627640 1 -- 172.21.15.78:0/361010690 shutdown_connections 2024-04-25T01:45:03.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.771+0000 7f475e627640 1 -- 172.21.15.78:0/361010690 wait complete. 2024-04-25T01:45:03.775 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:45:03.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.887+0000 7f649ba45640 1 -- 172.21.15.78:0/2089615066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6494102980 msgr2=0x7f6494102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.887+0000 7f649ba45640 1 --2- 172.21.15.78:0/2089615066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6494102980 0x7f6494102e00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6484009900 tx=0x7f648402a410 comp rx=0 tx=0).stop 2024-04-25T01:45:03.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.887+0000 7f649ba45640 1 -- 172.21.15.78:0/2089615066 shutdown_connections 2024-04-25T01:45:03.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.887+0000 7f649ba45640 1 --2- 172.21.15.78:0/2089615066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6494103340 0x7f6494109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.887+0000 7f649ba45640 1 --2- 172.21.15.78:0/2089615066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6494102980 0x7f6494102e00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.887+0000 7f649ba45640 1 --2- 172.21.15.78:0/2089615066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6494101790 0x7f6494101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.887+0000 7f649ba45640 1 -- 172.21.15.78:0/2089615066 >> 172.21.15.78:0/2089615066 conn(0x7f64940fcd20 msgr2=0x7f64940ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:03.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.887+0000 7f649ba45640 1 -- 172.21.15.78:0/2089615066 shutdown_connections 2024-04-25T01:45:03.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.887+0000 7f649ba45640 1 -- 172.21.15.78:0/2089615066 wait complete. 2024-04-25T01:45:03.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.887+0000 7f649ba45640 1 Processor -- start 2024-04-25T01:45:03.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f649ba45640 1 -- start start 2024-04-25T01:45:03.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f649ba45640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6494101790 0x7f6494199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:03.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f649ba45640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6494102980 0x7f649419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:03.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f649ba45640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6494103340 0x7f64941a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:03.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f649ba45640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6494074f60 con 0x7f6494103340 2024-04-25T01:45:03.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f649ba45640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6494074de0 con 0x7f6494102980 2024-04-25T01:45:03.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f649ba45640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f64940750e0 con 0x7f6494101790 2024-04-25T01:45:03.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f6499fe2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6494103340 0x7f64941a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:03.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f6499fe2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6494103340 0x7f64941a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40850/0 (socket says 172.21.15.78:40850) 2024-04-25T01:45:03.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f6499fe2640 1 -- 172.21.15.78:0/3596806810 learned_addr learned my addr 172.21.15.78:0/3596806810 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:03.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f64997e1640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6494101790 0x7f6494199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:03.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f6498fe0640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6494102980 0x7f649419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:03.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f6499fe2640 1 -- 172.21.15.78:0/3596806810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6494101790 msgr2=0x7f6494199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f6499fe2640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6494101790 0x7f6494199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f6499fe2640 1 -- 172.21.15.78:0/3596806810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6494102980 msgr2=0x7f649419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:03.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f6499fe2640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6494102980 0x7f649419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:03.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f6499fe2640 1 -- 172.21.15.78:0/3596806810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64941a18f0 con 0x7f6494103340 2024-04-25T01:45:03.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f6498fe0640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6494102980 0x7f649419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:03.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f6499fe2640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6494103340 0x7f64941a1250 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f648c00eaa0 tx=0x7f648c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:03.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f648a7fc640 1 -- 172.21.15.78:0/3596806810 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c00cdb0 con 0x7f6494103340 2024-04-25T01:45:03.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.888+0000 7f648a7fc640 1 -- 172.21.15.78:0/3596806810 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f648c005020 con 0x7f6494103340 2024-04-25T01:45:03.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.889+0000 7f649ba45640 1 -- 172.21.15.78:0/3596806810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64941a1be0 con 0x7f6494103340 2024-04-25T01:45:03.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.889+0000 7f648a7fc640 1 -- 172.21.15.78:0/3596806810 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c0116b0 con 0x7f6494103340 2024-04-25T01:45:03.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.889+0000 7f649ba45640 1 -- 172.21.15.78:0/3596806810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64941a2130 con 0x7f6494103340 2024-04-25T01:45:03.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.890+0000 7f649ba45640 1 -- 172.21.15.78:0/3596806810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64941089e0 con 0x7f6494103340 2024-04-25T01:45:03.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.891+0000 7f648a7fc640 1 -- 172.21.15.78:0/3596806810 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f648c005560 con 0x7f6494103340 2024-04-25T01:45:03.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.891+0000 7f648a7fc640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f64800a2c10 0x7f64800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:03.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.891+0000 7f64997e1640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f64800a2c10 0x7f64800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:03.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.891+0000 7f648a7fc640 1 -- 172.21.15.78:0/3596806810 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f648c0cf570 con 0x7f6494103340 2024-04-25T01:45:03.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.892+0000 7f64997e1640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f64800a2c10 0x7f64800a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f649419ab80 tx=0x7f649000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:03.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:03.896+0000 7f648a7fc640 1 -- 172.21.15.78:0/3596806810 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f648c016030 con 0x7f6494103340 2024-04-25T01:45:04.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.074+0000 7f649ba45640 1 -- 172.21.15.78:0/3596806810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f64940623e0 con 0x7f6494103340 2024-04-25T01:45:04.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.074+0000 7f648a7fc640 1 -- 172.21.15.78:0/3596806810 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f648c09c190 con 0x7f6494103340 2024-04-25T01:45:04.068 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:45:04.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.076+0000 7f649ba45640 1 -- 172.21.15.78:0/3596806810 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f64800a2c10 msgr2=0x7f64800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.076+0000 7f649ba45640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f64800a2c10 0x7f64800a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f649419ab80 tx=0x7f649000a2d0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.076+0000 7f649ba45640 1 -- 172.21.15.78:0/3596806810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6494103340 msgr2=0x7f64941a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.076+0000 7f649ba45640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6494103340 0x7f64941a1250 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f648c00eaa0 tx=0x7f648c00ef60 comp rx=0 tx=0).stop 2024-04-25T01:45:04.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.076+0000 7f649ba45640 1 -- 172.21.15.78:0/3596806810 shutdown_connections 2024-04-25T01:45:04.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.076+0000 7f649ba45640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f64800a2c10 0x7f64800a50d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.076+0000 7f649ba45640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6494103340 0x7f64941a1250 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.076+0000 7f649ba45640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6494102980 0x7f649419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.076+0000 7f649ba45640 1 --2- 172.21.15.78:0/3596806810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6494101790 0x7f6494199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.076+0000 7f649ba45640 1 -- 172.21.15.78:0/3596806810 >> 172.21.15.78:0/3596806810 conn(0x7f64940fcd20 msgr2=0x7f6494105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:04.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.077+0000 7f649ba45640 1 -- 172.21.15.78:0/3596806810 shutdown_connections 2024-04-25T01:45:04.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.077+0000 7f649ba45640 1 -- 172.21.15.78:0/3596806810 wait complete. 2024-04-25T01:45:04.080 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:03 smithi078 ceph-mon[29469]: pgmap v345: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T01:45:04.080 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:03 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3862399580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:45:04.080 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:03 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1719500078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:45:04.080 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:03 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/361010690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:45:04.081 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:45:04.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.185+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/258176958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a4103350 msgr2=0x7fd5a4109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.185+0000 7fd5ac3d5640 1 --2- 172.21.15.78:0/258176958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a4103350 0x7fd5a4109be0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009900 tx=0x7fd59c02a410 comp rx=0 tx=0).stop 2024-04-25T01:45:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.185+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/258176958 shutdown_connections 2024-04-25T01:45:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.185+0000 7fd5ac3d5640 1 --2- 172.21.15.78:0/258176958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a4103350 0x7fd5a4109be0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.185+0000 7fd5ac3d5640 1 --2- 172.21.15.78:0/258176958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a4102990 0x7fd5a4102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.185+0000 7fd5ac3d5640 1 --2- 172.21.15.78:0/258176958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a41017a0 0x7fd5a4101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/258176958 >> 172.21.15.78:0/258176958 conn(0x7fd5a40fccf0 msgr2=0x7fd5a40ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/258176958 shutdown_connections 2024-04-25T01:45:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/258176958 wait complete. 2024-04-25T01:45:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5ac3d5640 1 Processor -- start 2024-04-25T01:45:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5ac3d5640 1 -- start start 2024-04-25T01:45:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5ac3d5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a41017a0 0x7fd5a4197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5ac3d5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a4102990 0x7fd5a41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:04.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5ac3d5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a4103350 0x7fd5a419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5ac3d5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a410d510 con 0x7fd5a41017a0 2024-04-25T01:45:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5ac3d5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a410d390 con 0x7fd5a4102990 2024-04-25T01:45:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5ac3d5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a410d690 con 0x7fd5a4103350 2024-04-25T01:45:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5aa171640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a41017a0 0x7fd5a4197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5aa171640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a41017a0 0x7fd5a4197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40862/0 (socket says 172.21.15.78:40862) 2024-04-25T01:45:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5aa171640 1 -- 172.21.15.78:0/4197147386 learned_addr learned my addr 172.21.15.78:0/4197147386 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.186+0000 7fd5a9970640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a4102990 0x7fd5a41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.187+0000 7fd5aa972640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a4103350 0x7fd5a419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.187+0000 7fd5aa171640 1 -- 172.21.15.78:0/4197147386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a4103350 msgr2=0x7fd5a419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.187+0000 7fd5aa171640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a4103350 0x7fd5a419f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.187+0000 7fd5aa171640 1 -- 172.21.15.78:0/4197147386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a4102990 msgr2=0x7fd5a41980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.187+0000 7fd5aa171640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a4102990 0x7fd5a41980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.187+0000 7fd5aa171640 1 -- 172.21.15.78:0/4197147386 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a419f7d0 con 0x7fd5a41017a0 2024-04-25T01:45:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.187+0000 7fd5a9970640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a4102990 0x7fd5a41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.187+0000 7fd5aa171640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a41017a0 0x7fd5a4197bb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000b9c0 tx=0x7fd5a000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.187+0000 7fd59b7fe640 1 -- 172.21.15.78:0/4197147386 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0019070 con 0x7fd5a41017a0 2024-04-25T01:45:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.187+0000 7fd59b7fe640 1 -- 172.21.15.78:0/4197147386 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5a000cba0 con 0x7fd5a41017a0 2024-04-25T01:45:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.187+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/4197147386 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a419fac0 con 0x7fd5a41017a0 2024-04-25T01:45:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.187+0000 7fd59b7fe640 1 -- 172.21.15.78:0/4197147386 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0012640 con 0x7fd5a41017a0 2024-04-25T01:45:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.187+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/4197147386 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a41a0010 con 0x7fd5a41017a0 2024-04-25T01:45:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.188+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/4197147386 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a4108b10 con 0x7fd5a41017a0 2024-04-25T01:45:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.189+0000 7fd59b7fe640 1 -- 172.21.15.78:0/4197147386 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd5a001c070 con 0x7fd5a41017a0 2024-04-25T01:45:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.190+0000 7fd59b7fe640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd5900a2b90 0x7fd5900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.190+0000 7fd5a9970640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd5900a2b90 0x7fd5900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.190+0000 7fd59b7fe640 1 -- 172.21.15.78:0/4197147386 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fd5a00ce9e0 con 0x7fd5a41017a0 2024-04-25T01:45:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.190+0000 7fd5a9970640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd5900a2b90 0x7fd5900a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd5a41990c0 tx=0x7fd594008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.194+0000 7fd59b7fe640 1 -- 172.21.15.78:0/4197147386 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd5a009b600 con 0x7fd5a41017a0 2024-04-25T01:45:04.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:03 smithi152 ceph-mon[32465]: pgmap v345: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T01:45:04.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:03 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3862399580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:45:04.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:03 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1719500078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:45:04.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:03 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/361010690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:45:04.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:03 smithi196 ceph-mon[32978]: pgmap v345: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T01:45:04.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:03 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3862399580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:45:04.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:03 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1719500078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:45:04.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:03 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/361010690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:45:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.403+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/4197147386 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5a402d170 con 0x7fd5a41017a0 2024-04-25T01:45:04.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.404+0000 7fd59b7fe640 1 -- 172.21.15.78:0/4197147386 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7fd5a000cd50 con 0x7fd5a41017a0 2024-04-25T01:45:04.397 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:04.397 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:43:17.355192+0000","modified":"2024-04-25T01:43:44.435289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24455,"mds_2":24448,"mds_3":24392,"mds_4":24379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1337519666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1337519666},{"type":"v1","addr":"172.21.15.78:6839","nonce":1337519666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1963410683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1963410683},{"type":"v1","addr":"172.21.15.78:6841","nonce":1963410683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24379":{"gid":24379,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1969344843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1969344843},{"type":"v1","addr":"172.21.15.152:6839","nonce":1969344843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24379,24491,24455,24392,24448]},"id":1}]} 2024-04-25T01:45:04.398 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-04-25T01:45:04.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.406+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/4197147386 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd5900a2b90 msgr2=0x7fd5900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.406+0000 7fd5ac3d5640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd5900a2b90 0x7fd5900a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd5a41990c0 tx=0x7fd594008040 comp rx=0 tx=0).stop 2024-04-25T01:45:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.406+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/4197147386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a41017a0 msgr2=0x7fd5a4197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.406+0000 7fd5ac3d5640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a41017a0 0x7fd5a4197bb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000b9c0 tx=0x7fd5a000be80 comp rx=0 tx=0).stop 2024-04-25T01:45:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.406+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/4197147386 shutdown_connections 2024-04-25T01:45:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.406+0000 7fd5ac3d5640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5a4103350 0x7fd5a419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.406+0000 7fd5ac3d5640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a4102990 0x7fd5a41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.406+0000 7fd5ac3d5640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd5900a2b90 0x7fd5900a5050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.406+0000 7fd5ac3d5640 1 --2- 172.21.15.78:0/4197147386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a41017a0 0x7fd5a4197bb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.406+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/4197147386 >> 172.21.15.78:0/4197147386 conn(0x7fd5a40fccf0 msgr2=0x7fd5a4105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.406+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/4197147386 shutdown_connections 2024-04-25T01:45:04.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.406+0000 7fd5ac3d5640 1 -- 172.21.15.78:0/4197147386 wait complete. 2024-04-25T01:45:04.412 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:45:04.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.517+0000 7f4d46f32640 1 -- 172.21.15.78:0/2462397820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d40100f70 msgr2=0x7f4d401013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.517+0000 7f4d46f32640 1 --2- 172.21.15.78:0/2462397820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d40100f70 0x7f4d401013f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009900 tx=0x7f4d2c02a410 comp rx=0 tx=0).stop 2024-04-25T01:45:04.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.517+0000 7f4d46f32640 1 -- 172.21.15.78:0/2462397820 shutdown_connections 2024-04-25T01:45:04.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.517+0000 7f4d46f32640 1 --2- 172.21.15.78:0/2462397820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d40101930 0x7f4d40108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.517+0000 7f4d46f32640 1 --2- 172.21.15.78:0/2462397820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d40100f70 0x7f4d401013f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.517+0000 7f4d46f32640 1 --2- 172.21.15.78:0/2462397820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d400ffe20 0x7f4d40100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.517+0000 7f4d46f32640 1 -- 172.21.15.78:0/2462397820 >> 172.21.15.78:0/2462397820 conn(0x7f4d400fb3b0 msgr2=0x7f4d400fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:04.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.517+0000 7f4d46f32640 1 -- 172.21.15.78:0/2462397820 shutdown_connections 2024-04-25T01:45:04.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.517+0000 7f4d46f32640 1 -- 172.21.15.78:0/2462397820 wait complete. 2024-04-25T01:45:04.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.517+0000 7f4d46f32640 1 Processor -- start 2024-04-25T01:45:04.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d46f32640 1 -- start start 2024-04-25T01:45:04.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d46f32640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d400ffe20 0x7f4d401969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:04.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d46f32640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d40100f70 0x7f4d40196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:04.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d46f32640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d40101930 0x7f4d4019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:04.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d46f32640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4010c330 con 0x7f4d400ffe20 2024-04-25T01:45:04.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d46f32640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4010c1b0 con 0x7f4d40100f70 2024-04-25T01:45:04.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d46f32640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4010c4b0 con 0x7f4d40101930 2024-04-25T01:45:04.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d44cce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d400ffe20 0x7f4d401969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:04.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d44cce640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d400ffe20 0x7f4d401969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40866/0 (socket says 172.21.15.78:40866) 2024-04-25T01:45:04.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d44cce640 1 -- 172.21.15.78:0/3955211529 learned_addr learned my addr 172.21.15.78:0/3955211529 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:04.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d3ffff640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d40100f70 0x7f4d40196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:04.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d454cf640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d40101930 0x7f4d4019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:04.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d44cce640 1 -- 172.21.15.78:0/3955211529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d40101930 msgr2=0x7f4d4019df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d44cce640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d40101930 0x7f4d4019df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d44cce640 1 -- 172.21.15.78:0/3955211529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d40100f70 msgr2=0x7f4d40196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d44cce640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d40100f70 0x7f4d40196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d44cce640 1 -- 172.21.15.78:0/3955211529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4019e5c0 con 0x7f4d400ffe20 2024-04-25T01:45:04.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d454cf640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d40101930 0x7f4d4019df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:04.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d44cce640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d400ffe20 0x7f4d401969a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400d950 tx=0x7f4d3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:04.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d3dffb640 1 -- 172.21.15.78:0/3955211529 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34017070 con 0x7f4d400ffe20 2024-04-25T01:45:04.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d3dffb640 1 -- 172.21.15.78:0/3955211529 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d34009c30 con 0x7f4d400ffe20 2024-04-25T01:45:04.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d46f32640 1 -- 172.21.15.78:0/3955211529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4019e8b0 con 0x7f4d400ffe20 2024-04-25T01:45:04.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.518+0000 7f4d3dffb640 1 -- 172.21.15.78:0/3955211529 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3400f6b0 con 0x7f4d400ffe20 2024-04-25T01:45:04.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.519+0000 7f4d46f32640 1 -- 172.21.15.78:0/3955211529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4019edd0 con 0x7f4d400ffe20 2024-04-25T01:45:04.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.519+0000 7f4d46f32640 1 -- 172.21.15.78:0/3955211529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d401071b0 con 0x7f4d400ffe20 2024-04-25T01:45:04.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.520+0000 7f4d3dffb640 1 -- 172.21.15.78:0/3955211529 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d340109d0 con 0x7f4d400ffe20 2024-04-25T01:45:04.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.521+0000 7f4d3dffb640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4d300a2b90 0x7f4d300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:04.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.521+0000 7f4d3ffff640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4d300a2b90 0x7f4d300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:04.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.521+0000 7f4d3dffb640 1 -- 172.21.15.78:0/3955211529 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f4d340cf0c0 con 0x7f4d400ffe20 2024-04-25T01:45:04.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.521+0000 7f4d3ffff640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4d300a2b90 0x7f4d300a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c006010 tx=0x7f4d2c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:04.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.525+0000 7f4d3dffb640 1 -- 172.21.15.78:0/3955211529 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4d34009de0 con 0x7f4d400ffe20 2024-04-25T01:45:04.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.703+0000 7f4d46f32640 1 -- 172.21.15.78:0/3955211529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d40062340 con 0x7f4d400ffe20 2024-04-25T01:45:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.705+0000 7f4d3dffb640 1 -- 172.21.15.78:0/3955211529 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24592 (secure 0 0 0) 0x7f4d3409bce0 con 0x7f4d400ffe20 2024-04-25T01:45:04.698 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:04.699 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:44:18.500577+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:45:04.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.707+0000 7f4d46f32640 1 -- 172.21.15.78:0/3955211529 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4d300a2b90 msgr2=0x7f4d300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.707+0000 7f4d46f32640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4d300a2b90 0x7f4d300a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c006010 tx=0x7f4d2c000bc0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.707+0000 7f4d46f32640 1 -- 172.21.15.78:0/3955211529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d400ffe20 msgr2=0x7f4d401969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.707+0000 7f4d46f32640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d400ffe20 0x7f4d401969a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400d950 tx=0x7f4d3400de10 comp rx=0 tx=0).stop 2024-04-25T01:45:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.707+0000 7f4d46f32640 1 -- 172.21.15.78:0/3955211529 shutdown_connections 2024-04-25T01:45:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.707+0000 7f4d46f32640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d40101930 0x7f4d4019df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.707+0000 7f4d46f32640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d40100f70 0x7f4d40196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.707+0000 7f4d46f32640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4d300a2b90 0x7f4d300a5050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.707+0000 7f4d46f32640 1 --2- 172.21.15.78:0/3955211529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d400ffe20 0x7f4d401969a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.707+0000 7f4d46f32640 1 -- 172.21.15.78:0/3955211529 >> 172.21.15.78:0/3955211529 conn(0x7f4d400fb3b0 msgr2=0x7f4d40104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.707+0000 7f4d46f32640 1 -- 172.21.15.78:0/3955211529 shutdown_connections 2024-04-25T01:45:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.707+0000 7f4d46f32640 1 -- 172.21.15.78:0/3955211529 wait complete. 2024-04-25T01:45:04.712 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:38:39.136418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:43:15.613881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:43:16.600270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:43:19.650457+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:45:04.712 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:45:04.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.821+0000 7fa750b5c640 1 -- 172.21.15.78:0/3737415445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa74c101890 msgr2=0x7fa74c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.821+0000 7fa750b5c640 1 --2- 172.21.15.78:0/3737415445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa74c101890 0x7fa74c101d10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa734009900 tx=0x7fa73402a410 comp rx=0 tx=0).stop 2024-04-25T01:45:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.821+0000 7fa750b5c640 1 -- 172.21.15.78:0/3737415445 shutdown_connections 2024-04-25T01:45:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.821+0000 7fa750b5c640 1 --2- 172.21.15.78:0/3737415445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa74c102250 0x7fa74c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.821+0000 7fa750b5c640 1 --2- 172.21.15.78:0/3737415445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa74c101890 0x7fa74c101d10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.821+0000 7fa750b5c640 1 --2- 172.21.15.78:0/3737415445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa74c1006a0 0x7fa74c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.821+0000 7fa750b5c640 1 -- 172.21.15.78:0/3737415445 >> 172.21.15.78:0/3737415445 conn(0x7fa74c0fbc30 msgr2=0x7fa74c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.821+0000 7fa750b5c640 1 -- 172.21.15.78:0/3737415445 shutdown_connections 2024-04-25T01:45:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.821+0000 7fa750b5c640 1 -- 172.21.15.78:0/3737415445 wait complete. 2024-04-25T01:45:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa750b5c640 1 Processor -- start 2024-04-25T01:45:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa750b5c640 1 -- start start 2024-04-25T01:45:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa750b5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa74c1006a0 0x7fa74c196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa750b5c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa74c101890 0x7fa74c197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa750b5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa74c102250 0x7fa74c19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa750b5c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c10c460 con 0x7fa74c102250 2024-04-25T01:45:04.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa750b5c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c10c2e0 con 0x7fa74c1006a0 2024-04-25T01:45:04.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa750b5c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c10c5e0 con 0x7fa74c101890 2024-04-25T01:45:04.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa74ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa74c102250 0x7fa74c19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:04.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa74ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa74c102250 0x7fa74c19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40880/0 (socket says 172.21.15.78:40880) 2024-04-25T01:45:04.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa74ad9d640 1 -- 172.21.15.78:0/2267612632 learned_addr learned my addr 172.21.15.78:0/2267612632 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:04.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa74a59c640 1 --2- 172.21.15.78:0/2267612632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa74c1006a0 0x7fa74c196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:04.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa743fff640 1 --2- 172.21.15.78:0/2267612632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa74c101890 0x7fa74c197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:04.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa74ad9d640 1 -- 172.21.15.78:0/2267612632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa74c101890 msgr2=0x7fa74c197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa74ad9d640 1 --2- 172.21.15.78:0/2267612632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa74c101890 0x7fa74c197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa74ad9d640 1 -- 172.21.15.78:0/2267612632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa74c1006a0 msgr2=0x7fa74c196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:04.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa74ad9d640 1 --2- 172.21.15.78:0/2267612632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa74c1006a0 0x7fa74c196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:04.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.822+0000 7fa74ad9d640 1 -- 172.21.15.78:0/2267612632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74c19e770 con 0x7fa74c102250 2024-04-25T01:45:04.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.823+0000 7fa74ad9d640 1 --2- 172.21.15.78:0/2267612632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa74c102250 0x7fa74c19e100 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa74400b970 tx=0x7fa74400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:04.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.823+0000 7fa7437fe640 1 -- 172.21.15.78:0/2267612632 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744019070 con 0x7fa74c102250 2024-04-25T01:45:04.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.823+0000 7fa7437fe640 1 -- 172.21.15.78:0/2267612632 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74400cb40 con 0x7fa74c102250 2024-04-25T01:45:04.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.823+0000 7fa7437fe640 1 -- 172.21.15.78:0/2267612632 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744007650 con 0x7fa74c102250 2024-04-25T01:45:04.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.823+0000 7fa750b5c640 1 -- 172.21.15.78:0/2267612632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74c19ea60 con 0x7fa74c102250 2024-04-25T01:45:04.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.823+0000 7fa750b5c640 1 -- 172.21.15.78:0/2267612632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74c19efb0 con 0x7fa74c102250 2024-04-25T01:45:04.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.824+0000 7fa750b5c640 1 -- 172.21.15.78:0/2267612632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa74c107a90 con 0x7fa74c102250 2024-04-25T01:45:04.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.825+0000 7fa7437fe640 1 -- 172.21.15.78:0/2267612632 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa74401c070 con 0x7fa74c102250 2024-04-25T01:45:04.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.826+0000 7fa7437fe640 1 --2- 172.21.15.78:0/2267612632 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa7380a2b90 0x7fa7380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:04.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.826+0000 7fa74a59c640 1 --2- 172.21.15.78:0/2267612632 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa7380a2b90 0x7fa7380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:04.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.826+0000 7fa7437fe640 1 -- 172.21.15.78:0/2267612632 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fa7440cea10 con 0x7fa74c102250 2024-04-25T01:45:04.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.826+0000 7fa74a59c640 1 --2- 172.21.15.78:0/2267612632 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa7380a2b90 0x7fa7380a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa74c1016f0 tx=0x7fa73c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:04.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:04.830+0000 7fa7437fe640 1 -- 172.21.15.78:0/2267612632 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa74409b630 con 0x7fa74c102250 2024-04-25T01:45:05.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.007+0000 7fa750b5c640 1 -- 172.21.15.78:0/2267612632 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa74c062340 con 0x7fa74c102250 2024-04-25T01:45:05.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.007+0000 7fa7437fe640 1 -- 172.21.15.78:0/2267612632 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fa74409af30 con 0x7fa74c102250 2024-04-25T01:45:05.001 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T01:45:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.009+0000 7fa750b5c640 1 -- 172.21.15.78:0/2267612632 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa7380a2b90 msgr2=0x7fa7380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.009+0000 7fa750b5c640 1 --2- 172.21.15.78:0/2267612632 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa7380a2b90 0x7fa7380a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa74c1016f0 tx=0x7fa73c008040 comp rx=0 tx=0).stop 2024-04-25T01:45:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.009+0000 7fa750b5c640 1 -- 172.21.15.78:0/2267612632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa74c102250 msgr2=0x7fa74c19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.009+0000 7fa750b5c640 1 --2- 172.21.15.78:0/2267612632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa74c102250 0x7fa74c19e100 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa74400b970 tx=0x7fa74400be30 comp rx=0 tx=0).stop 2024-04-25T01:45:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.009+0000 7fa750b5c640 1 -- 172.21.15.78:0/2267612632 shutdown_connections 2024-04-25T01:45:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.009+0000 7fa750b5c640 1 --2- 172.21.15.78:0/2267612632 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa7380a2b90 0x7fa7380a5050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.009+0000 7fa750b5c640 1 --2- 172.21.15.78:0/2267612632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa74c102250 0x7fa74c19e100 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.009+0000 7fa750b5c640 1 --2- 172.21.15.78:0/2267612632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa74c101890 0x7fa74c197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.009+0000 7fa750b5c640 1 --2- 172.21.15.78:0/2267612632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa74c1006a0 0x7fa74c196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.009+0000 7fa750b5c640 1 -- 172.21.15.78:0/2267612632 >> 172.21.15.78:0/2267612632 conn(0x7fa74c0fbc30 msgr2=0x7fa74c104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.010+0000 7fa750b5c640 1 -- 172.21.15.78:0/2267612632 shutdown_connections 2024-04-25T01:45:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.010+0000 7fa750b5c640 1 -- 172.21.15.78:0/2267612632 wait complete. 2024-04-25T01:45:05.014 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:45:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.122+0000 7f4110963640 1 -- 172.21.15.78:0/2375939765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f410c0ffe10 msgr2=0x7f410c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.122+0000 7f4110963640 1 --2- 172.21.15.78:0/2375939765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f410c0ffe10 0x7f410c100230 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4104009ca0 tx=0x7f410402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.122+0000 7f4110963640 1 -- 172.21.15.78:0/2375939765 shutdown_connections 2024-04-25T01:45:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.122+0000 7f4110963640 1 --2- 172.21.15.78:0/2375939765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f410c101920 0x7f410c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.122+0000 7f4110963640 1 --2- 172.21.15.78:0/2375939765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f410c100f60 0x7f410c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.122+0000 7f4110963640 1 --2- 172.21.15.78:0/2375939765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f410c0ffe10 0x7f410c100230 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.122+0000 7f4110963640 1 -- 172.21.15.78:0/2375939765 >> 172.21.15.78:0/2375939765 conn(0x7f410c0fb380 msgr2=0x7f410c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.122+0000 7f4110963640 1 -- 172.21.15.78:0/2375939765 shutdown_connections 2024-04-25T01:45:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.122+0000 7f4110963640 1 -- 172.21.15.78:0/2375939765 wait complete. 2024-04-25T01:45:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f4110963640 1 Processor -- start 2024-04-25T01:45:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f4110963640 1 -- start start 2024-04-25T01:45:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f4110963640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f410c0ffe10 0x7f410c1097a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:05.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f4110963640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f410c100f60 0x7f410c19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f4110963640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f410c101920 0x7f410c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f4110963640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f410c10c430 con 0x7f410c101920 2024-04-25T01:45:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f4110963640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f410c10c2b0 con 0x7f410c100f60 2024-04-25T01:45:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f4110963640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f410c10c5b0 con 0x7f410c0ffe10 2024-04-25T01:45:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f410ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f410c101920 0x7f410c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f410ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f410c101920 0x7f410c19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40884/0 (socket says 172.21.15.78:40884) 2024-04-25T01:45:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f410ad9d640 1 -- 172.21.15.78:0/2919674270 learned_addr learned my addr 172.21.15.78:0/2919674270 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f410a59c640 1 --2- 172.21.15.78:0/2919674270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f410c0ffe10 0x7f410c1097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f410ad9d640 1 -- 172.21.15.78:0/2919674270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f410c0ffe10 msgr2=0x7f410c1097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f4109d9b640 1 --2- 172.21.15.78:0/2919674270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f410c100f60 0x7f410c19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f410ad9d640 1 --2- 172.21.15.78:0/2919674270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f410c0ffe10 0x7f410c1097a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f410ad9d640 1 -- 172.21.15.78:0/2919674270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f410c100f60 msgr2=0x7f410c19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f410ad9d640 1 --2- 172.21.15.78:0/2919674270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f410c100f60 0x7f410c19c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f410ad9d640 1 -- 172.21.15.78:0/2919674270 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410c19f8b0 con 0x7f410c101920 2024-04-25T01:45:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.123+0000 7f410ad9d640 1 --2- 172.21.15.78:0/2919674270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f410c101920 0x7f410c19efa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009fd0 tx=0x7f40fc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.124+0000 7f41037fe640 1 -- 172.21.15.78:0/2919674270 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc011830 con 0x7f410c101920 2024-04-25T01:45:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.124+0000 7f41037fe640 1 -- 172.21.15.78:0/2919674270 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40fc016070 con 0x7f410c101920 2024-04-25T01:45:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.124+0000 7f4110963640 1 -- 172.21.15.78:0/2919674270 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f410c19fb40 con 0x7f410c101920 2024-04-25T01:45:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.124+0000 7f41037fe640 1 -- 172.21.15.78:0/2919674270 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc00f2e0 con 0x7f410c101920 2024-04-25T01:45:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.124+0000 7f4110963640 1 -- 172.21.15.78:0/2919674270 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f410c1a0110 con 0x7f410c101920 2024-04-25T01:45:05.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.125+0000 7f4110963640 1 -- 172.21.15.78:0/2919674270 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410c1071f0 con 0x7f410c101920 2024-04-25T01:45:05.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.126+0000 7f41037fe640 1 -- 172.21.15.78:0/2919674270 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f40fc0119d0 con 0x7f410c101920 2024-04-25T01:45:05.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.126+0000 7f41037fe640 1 --2- 172.21.15.78:0/2919674270 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40f80a2b90 0x7f40f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:05.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.126+0000 7f410a59c640 1 --2- 172.21.15.78:0/2919674270 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40f80a2b90 0x7f40f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:05.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.126+0000 7f41037fe640 1 -- 172.21.15.78:0/2919674270 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f40fc0ce9a0 con 0x7f410c101920 2024-04-25T01:45:05.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.127+0000 7f410a59c640 1 --2- 172.21.15.78:0/2919674270 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40f80a2b90 0x7f40f80a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4104006010 tx=0x7f4104000f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:05.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.131+0000 7f41037fe640 1 -- 172.21.15.78:0/2919674270 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f40fc09b5c0 con 0x7f410c101920 2024-04-25T01:45:05.257 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:05 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3596806810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:45:05.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:05 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/4197147386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:05.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:05 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3955211529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:05.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.304+0000 7f4110963640 1 -- 172.21.15.78:0/2919674270 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f410c062340 con 0x7f410c101920 2024-04-25T01:45:05.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.305+0000 7f41037fe640 1 -- 172.21.15.78:0/2919674270 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f40fc09aec0 con 0x7f410c101920 2024-04-25T01:45:05.298 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:45:05.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:05 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3596806810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:45:05.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:05 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/4197147386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:05.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:05 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3955211529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:05.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.307+0000 7f4110963640 1 -- 172.21.15.78:0/2919674270 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40f80a2b90 msgr2=0x7f40f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.307+0000 7f4110963640 1 --2- 172.21.15.78:0/2919674270 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40f80a2b90 0x7f40f80a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4104006010 tx=0x7f4104000f40 comp rx=0 tx=0).stop 2024-04-25T01:45:05.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.307+0000 7f4110963640 1 -- 172.21.15.78:0/2919674270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f410c101920 msgr2=0x7f410c19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.307+0000 7f4110963640 1 --2- 172.21.15.78:0/2919674270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f410c101920 0x7f410c19efa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009fd0 tx=0x7f40fc00b8e0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.307+0000 7f4110963640 1 -- 172.21.15.78:0/2919674270 shutdown_connections 2024-04-25T01:45:05.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.307+0000 7f4110963640 1 --2- 172.21.15.78:0/2919674270 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40f80a2b90 0x7f40f80a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.307+0000 7f4110963640 1 --2- 172.21.15.78:0/2919674270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f410c101920 0x7f410c19efa0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.307+0000 7f4110963640 1 --2- 172.21.15.78:0/2919674270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f410c100f60 0x7f410c19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.307+0000 7f4110963640 1 --2- 172.21.15.78:0/2919674270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f410c0ffe10 0x7f410c1097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.307+0000 7f4110963640 1 -- 172.21.15.78:0/2919674270 >> 172.21.15.78:0/2919674270 conn(0x7f410c0fb380 msgr2=0x7f410c1043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:05.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.307+0000 7f4110963640 1 -- 172.21.15.78:0/2919674270 shutdown_connections 2024-04-25T01:45:05.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.307+0000 7f4110963640 1 -- 172.21.15.78:0/2919674270 wait complete. 2024-04-25T01:45:05.311 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:45:05.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:05 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3596806810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:45:05.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:05 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/4197147386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:05.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:05 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3955211529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.419+0000 7f470f934640 1 -- 172.21.15.78:0/122384636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4708101870 msgr2=0x7f4708101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.419+0000 7f470f934640 1 --2- 172.21.15.78:0/122384636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4708101870 0x7f4708101cf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009900 tx=0x7f46f802a410 comp rx=0 tx=0).stop 2024-04-25T01:45:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.419+0000 7f470f934640 1 -- 172.21.15.78:0/122384636 shutdown_connections 2024-04-25T01:45:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.419+0000 7f470f934640 1 --2- 172.21.15.78:0/122384636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4708102230 0x7f4708108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.419+0000 7f470f934640 1 --2- 172.21.15.78:0/122384636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4708101870 0x7f4708101cf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.419+0000 7f470f934640 1 --2- 172.21.15.78:0/122384636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708100680 0x7f4708100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.419+0000 7f470f934640 1 -- 172.21.15.78:0/122384636 >> 172.21.15.78:0/122384636 conn(0x7f47080fbbf0 msgr2=0x7f47080fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:05.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.420+0000 7f470f934640 1 -- 172.21.15.78:0/122384636 shutdown_connections 2024-04-25T01:45:05.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.420+0000 7f470f934640 1 -- 172.21.15.78:0/122384636 wait complete. 2024-04-25T01:45:05.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.420+0000 7f470f934640 1 Processor -- start 2024-04-25T01:45:05.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.420+0000 7f470f934640 1 -- start start 2024-04-25T01:45:05.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.420+0000 7f470f934640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4708100680 0x7f4708069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:05.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.420+0000 7f470f934640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4708101870 0x7f470819c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:05.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.420+0000 7f470f934640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708102230 0x7f470819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:05.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.420+0000 7f470f934640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f470810c2e0 con 0x7f4708101870 2024-04-25T01:45:05.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.420+0000 7f470f934640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f470810c160 con 0x7f4708102230 2024-04-25T01:45:05.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.420+0000 7f470f934640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f470810c460 con 0x7f4708100680 2024-04-25T01:45:05.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.420+0000 7f470cecf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4708101870 0x7f470819c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:05.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.420+0000 7f470cecf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4708101870 0x7f470819c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40908/0 (socket says 172.21.15.78:40908) 2024-04-25T01:45:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.420+0000 7f470cecf640 1 -- 172.21.15.78:0/1584901684 learned_addr learned my addr 172.21.15.78:0/1584901684 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.420+0000 7f470d6d0640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4708100680 0x7f4708069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.421+0000 7f470ded1640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708102230 0x7f470819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.421+0000 7f470cecf640 1 -- 172.21.15.78:0/1584901684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4708100680 msgr2=0x7f4708069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.421+0000 7f470cecf640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4708100680 0x7f4708069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.421+0000 7f470cecf640 1 -- 172.21.15.78:0/1584901684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708102230 msgr2=0x7f470819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.421+0000 7f470cecf640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708102230 0x7f470819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.421+0000 7f470cecf640 1 -- 172.21.15.78:0/1584901684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470819f680 con 0x7f4708101870 2024-04-25T01:45:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.421+0000 7f470ded1640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708102230 0x7f470819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.421+0000 7f470cecf640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4708101870 0x7f470819c780 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009230 tx=0x7f46f8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.421+0000 7f46fe7fc640 1 -- 172.21.15.78:0/1584901684 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f803f030 con 0x7f4708101870 2024-04-25T01:45:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.421+0000 7f46fe7fc640 1 -- 172.21.15.78:0/1584901684 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46f8003ce0 con 0x7f4708101870 2024-04-25T01:45:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.421+0000 7f46fe7fc640 1 -- 172.21.15.78:0/1584901684 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8035070 con 0x7f4708101870 2024-04-25T01:45:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.421+0000 7f470f934640 1 -- 172.21.15.78:0/1584901684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470819f910 con 0x7f4708101870 2024-04-25T01:45:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.421+0000 7f470f934640 1 -- 172.21.15.78:0/1584901684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470819fe00 con 0x7f4708101870 2024-04-25T01:45:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.422+0000 7f470f934640 1 -- 172.21.15.78:0/1584901684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47081079a0 con 0x7f4708101870 2024-04-25T01:45:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.423+0000 7f46fe7fc640 1 -- 172.21.15.78:0/1584901684 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f46f8042030 con 0x7f4708101870 2024-04-25T01:45:05.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.423+0000 7f46fe7fc640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f46f40a2b90 0x7f46f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:05.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.424+0000 7f470d6d0640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f46f40a2b90 0x7f46f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:05.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.424+0000 7f46fe7fc640 1 -- 172.21.15.78:0/1584901684 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f46f80efd70 con 0x7f4708101870 2024-04-25T01:45:05.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.424+0000 7f470d6d0640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f46f40a2b90 0x7f46f40a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f47081016d0 tx=0x7f4704007600 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:05.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.428+0000 7f46fe7fc640 1 -- 172.21.15.78:0/1584901684 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f46f80bc990 con 0x7f4708101870 2024-04-25T01:45:05.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.608+0000 7f470f934640 1 -- 172.21.15.78:0/1584901684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4708062340 con 0x7f4708101870 2024-04-25T01:45:05.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.609+0000 7f46fe7fc640 1 -- 172.21.15.78:0/1584901684 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f46f80bc290 con 0x7f4708101870 2024-04-25T01:45:05.602 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T01:45:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.610+0000 7f470f934640 1 -- 172.21.15.78:0/1584901684 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f46f40a2b90 msgr2=0x7f46f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.610+0000 7f470f934640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f46f40a2b90 0x7f46f40a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f47081016d0 tx=0x7f4704007600 comp rx=0 tx=0).stop 2024-04-25T01:45:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.610+0000 7f470f934640 1 -- 172.21.15.78:0/1584901684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4708101870 msgr2=0x7f470819c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.610+0000 7f470f934640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4708101870 0x7f470819c780 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009230 tx=0x7f46f8004360 comp rx=0 tx=0).stop 2024-04-25T01:45:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.611+0000 7f470f934640 1 -- 172.21.15.78:0/1584901684 shutdown_connections 2024-04-25T01:45:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.611+0000 7f470f934640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708102230 0x7f470819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.611+0000 7f470f934640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f46f40a2b90 0x7f46f40a5050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.611+0000 7f470f934640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4708101870 0x7f470819c780 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.611+0000 7f470f934640 1 --2- 172.21.15.78:0/1584901684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4708100680 0x7f4708069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.611+0000 7f470f934640 1 -- 172.21.15.78:0/1584901684 >> 172.21.15.78:0/1584901684 conn(0x7f47080fbbf0 msgr2=0x7f4708104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:05.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.611+0000 7f470f934640 1 -- 172.21.15.78:0/1584901684 shutdown_connections 2024-04-25T01:45:05.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.611+0000 7f470f934640 1 -- 172.21.15.78:0/1584901684 wait complete. 2024-04-25T01:45:05.615 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:45:05.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.724+0000 7fda7c5b6640 1 -- 172.21.15.78:0/4243733998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda741006a0 msgr2=0x7fda74100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.724+0000 7fda7c5b6640 1 --2- 172.21.15.78:0/4243733998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda741006a0 0x7fda74100ac0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fda70009900 tx=0x7fda7002a410 comp rx=0 tx=0).stop 2024-04-25T01:45:05.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.724+0000 7fda7c5b6640 1 -- 172.21.15.78:0/4243733998 shutdown_connections 2024-04-25T01:45:05.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.724+0000 7fda7c5b6640 1 --2- 172.21.15.78:0/4243733998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda74102250 0x7fda74108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.724+0000 7fda7c5b6640 1 --2- 172.21.15.78:0/4243733998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda74101890 0x7fda74101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.724+0000 7fda7c5b6640 1 --2- 172.21.15.78:0/4243733998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda741006a0 0x7fda74100ac0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.724+0000 7fda7c5b6640 1 -- 172.21.15.78:0/4243733998 >> 172.21.15.78:0/4243733998 conn(0x7fda740fbc10 msgr2=0x7fda740fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:05.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.724+0000 7fda7c5b6640 1 -- 172.21.15.78:0/4243733998 shutdown_connections 2024-04-25T01:45:05.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.724+0000 7fda7c5b6640 1 -- 172.21.15.78:0/4243733998 wait complete. 2024-04-25T01:45:05.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.725+0000 7fda7c5b6640 1 Processor -- start 2024-04-25T01:45:05.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.725+0000 7fda7c5b6640 1 -- start start 2024-04-25T01:45:05.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.725+0000 7fda7c5b6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda741006a0 0x7fda7406a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:05.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.725+0000 7fda7c5b6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda74101890 0x7fda7419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:05.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.725+0000 7fda7c5b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda74102250 0x7fda7419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:05.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.725+0000 7fda7c5b6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda7410c2f0 con 0x7fda74102250 2024-04-25T01:45:05.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.725+0000 7fda7c5b6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fda7410c170 con 0x7fda74101890 2024-04-25T01:45:05.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.725+0000 7fda7c5b6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fda7410c470 con 0x7fda741006a0 2024-04-25T01:45:05.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.725+0000 7fda7ab53640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda74102250 0x7fda7419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:05.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.725+0000 7fda7ab53640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda74102250 0x7fda7419ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40924/0 (socket says 172.21.15.78:40924) 2024-04-25T01:45:05.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.725+0000 7fda7ab53640 1 -- 172.21.15.78:0/3868783345 learned_addr learned my addr 172.21.15.78:0/3868783345 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:05.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.725+0000 7fda79b51640 1 --2- 172.21.15.78:0/3868783345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda74101890 0x7fda7419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:05.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.725+0000 7fda7a352640 1 --2- 172.21.15.78:0/3868783345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda741006a0 0x7fda7406a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:05.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.726+0000 7fda7ab53640 1 -- 172.21.15.78:0/3868783345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda741006a0 msgr2=0x7fda7406a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.726+0000 7fda7ab53640 1 --2- 172.21.15.78:0/3868783345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda741006a0 0x7fda7406a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.726+0000 7fda7ab53640 1 -- 172.21.15.78:0/3868783345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda74101890 msgr2=0x7fda7419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.726+0000 7fda7ab53640 1 --2- 172.21.15.78:0/3868783345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda74101890 0x7fda7419c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.726+0000 7fda7ab53640 1 -- 172.21.15.78:0/3868783345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7419f770 con 0x7fda74102250 2024-04-25T01:45:05.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.726+0000 7fda7ab53640 1 --2- 172.21.15.78:0/3868783345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda74102250 0x7fda7419ee60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00eaa0 tx=0x7fda6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:05.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.726+0000 7fda6b7fe640 1 -- 172.21.15.78:0/3868783345 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c00cdb0 con 0x7fda74102250 2024-04-25T01:45:05.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.726+0000 7fda6b7fe640 1 -- 172.21.15.78:0/3868783345 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda6c005020 con 0x7fda74102250 2024-04-25T01:45:05.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.726+0000 7fda7c5b6640 1 -- 172.21.15.78:0/3868783345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7419fa00 con 0x7fda74102250 2024-04-25T01:45:05.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.726+0000 7fda6b7fe640 1 -- 172.21.15.78:0/3868783345 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c0116b0 con 0x7fda74102250 2024-04-25T01:45:05.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.726+0000 7fda7c5b6640 1 -- 172.21.15.78:0/3868783345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7419ff00 con 0x7fda74102250 2024-04-25T01:45:05.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.727+0000 7fda7c5b6640 1 -- 172.21.15.78:0/3868783345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda74107a40 con 0x7fda74102250 2024-04-25T01:45:05.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.728+0000 7fda6b7fe640 1 -- 172.21.15.78:0/3868783345 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fda6c005560 con 0x7fda74102250 2024-04-25T01:45:05.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.728+0000 7fda6b7fe640 1 --2- 172.21.15.78:0/3868783345 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fda600a2b90 0x7fda600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:05.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.729+0000 7fda7a352640 1 --2- 172.21.15.78:0/3868783345 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fda600a2b90 0x7fda600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:05.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.729+0000 7fda6b7fe640 1 -- 172.21.15.78:0/3868783345 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fda6c0cf550 con 0x7fda74102250 2024-04-25T01:45:05.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.729+0000 7fda7a352640 1 --2- 172.21.15.78:0/3868783345 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fda600a2b90 0x7fda600a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fda70005ab0 tx=0x7fda70000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:05.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.733+0000 7fda6b7fe640 1 -- 172.21.15.78:0/3868783345 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fda6c016030 con 0x7fda74102250 2024-04-25T01:45:05.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.909+0000 7fda7c5b6640 1 -- 172.21.15.78:0/3868783345 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fda74062340 con 0x7fda74102250 2024-04-25T01:45:05.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.910+0000 7fda6b7fe640 1 -- 172.21.15.78:0/3868783345 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fda6c09bb20 con 0x7fda74102250 2024-04-25T01:45:05.903 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:45:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.912+0000 7fda7c5b6640 1 -- 172.21.15.78:0/3868783345 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fda600a2b90 msgr2=0x7fda600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.912+0000 7fda7c5b6640 1 --2- 172.21.15.78:0/3868783345 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fda600a2b90 0x7fda600a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fda70005ab0 tx=0x7fda70000bc0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.912+0000 7fda7c5b6640 1 -- 172.21.15.78:0/3868783345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda74102250 msgr2=0x7fda7419ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:05.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.912+0000 7fda7c5b6640 1 --2- 172.21.15.78:0/3868783345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda74102250 0x7fda7419ee60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00eaa0 tx=0x7fda6c00ef60 comp rx=0 tx=0).stop 2024-04-25T01:45:05.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.912+0000 7fda7c5b6640 1 -- 172.21.15.78:0/3868783345 shutdown_connections 2024-04-25T01:45:05.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.912+0000 7fda7c5b6640 1 --2- 172.21.15.78:0/3868783345 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fda600a2b90 0x7fda600a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.912+0000 7fda7c5b6640 1 --2- 172.21.15.78:0/3868783345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda74102250 0x7fda7419ee60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.912+0000 7fda7c5b6640 1 --2- 172.21.15.78:0/3868783345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda74101890 0x7fda7419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.912+0000 7fda7c5b6640 1 --2- 172.21.15.78:0/3868783345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda741006a0 0x7fda7406a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:05.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.912+0000 7fda7c5b6640 1 -- 172.21.15.78:0/3868783345 >> 172.21.15.78:0/3868783345 conn(0x7fda740fbc10 msgr2=0x7fda74104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:05.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.912+0000 7fda7c5b6640 1 -- 172.21.15.78:0/3868783345 shutdown_connections 2024-04-25T01:45:05.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:05.912+0000 7fda7c5b6640 1 -- 172.21.15.78:0/3868783345 wait complete. 2024-04-25T01:45:05.917 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:45:06.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.023+0000 7f364b8da640 1 -- 172.21.15.78:0/1141900983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3644102980 msgr2=0x7f3644102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.023+0000 7f364b8da640 1 --2- 172.21.15.78:0/1141900983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3644102980 0x7f3644102e00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3634009900 tx=0x7f363402a410 comp rx=0 tx=0).stop 2024-04-25T01:45:06.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.023+0000 7f364b8da640 1 -- 172.21.15.78:0/1141900983 shutdown_connections 2024-04-25T01:45:06.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.023+0000 7f364b8da640 1 --2- 172.21.15.78:0/1141900983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3644103340 0x7f3644109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.023+0000 7f364b8da640 1 --2- 172.21.15.78:0/1141900983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3644102980 0x7f3644102e00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.023+0000 7f364b8da640 1 --2- 172.21.15.78:0/1141900983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3644101790 0x7f3644101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.023+0000 7f364b8da640 1 -- 172.21.15.78:0/1141900983 >> 172.21.15.78:0/1141900983 conn(0x7f36440fcd20 msgr2=0x7f36440ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:06.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.023+0000 7f364b8da640 1 -- 172.21.15.78:0/1141900983 shutdown_connections 2024-04-25T01:45:06.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.023+0000 7f364b8da640 1 -- 172.21.15.78:0/1141900983 wait complete. 2024-04-25T01:45:06.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.023+0000 7f364b8da640 1 Processor -- start 2024-04-25T01:45:06.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.023+0000 7f364b8da640 1 -- start start 2024-04-25T01:45:06.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f364b8da640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3644101790 0x7f3644197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:06.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f364b8da640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3644102980 0x7f3644197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:06.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f364b8da640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3644103340 0x7f364419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:06.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f364b8da640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f364410d410 con 0x7f3644102980 2024-04-25T01:45:06.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f364b8da640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f364410d290 con 0x7f3644101790 2024-04-25T01:45:06.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f364b8da640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f364410d590 con 0x7f3644103340 2024-04-25T01:45:06.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f3648e75640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3644102980 0x7f3644197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:06.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f3648e75640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3644102980 0x7f3644197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40940/0 (socket says 172.21.15.78:40940) 2024-04-25T01:45:06.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f3648e75640 1 -- 172.21.15.78:0/3248817840 learned_addr learned my addr 172.21.15.78:0/3248817840 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:06.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f3649e77640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3644103340 0x7f364419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:06.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f3649676640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3644101790 0x7f3644197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:06.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f3648e75640 1 -- 172.21.15.78:0/3248817840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3644103340 msgr2=0x7f364419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f3648e75640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3644103340 0x7f364419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f3648e75640 1 -- 172.21.15.78:0/3248817840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3644101790 msgr2=0x7f3644197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f3648e75640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3644101790 0x7f3644197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f3648e75640 1 -- 172.21.15.78:0/3248817840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364419f6b0 con 0x7f3644102980 2024-04-25T01:45:06.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f3649e77640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3644103340 0x7f364419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:06.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f3648e75640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3644102980 0x7f3644197fd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3634009230 tx=0x7f3634004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:06.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f363a7fc640 1 -- 172.21.15.78:0/3248817840 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363403f030 con 0x7f3644102980 2024-04-25T01:45:06.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f363a7fc640 1 -- 172.21.15.78:0/3248817840 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3634003ce0 con 0x7f3644102980 2024-04-25T01:45:06.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f364b8da640 1 -- 172.21.15.78:0/3248817840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364419f940 con 0x7f3644102980 2024-04-25T01:45:06.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f363a7fc640 1 -- 172.21.15.78:0/3248817840 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634035070 con 0x7f3644102980 2024-04-25T01:45:06.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.024+0000 7f364b8da640 1 -- 172.21.15.78:0/3248817840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f364419fe30 con 0x7f3644102980 2024-04-25T01:45:06.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.025+0000 7f364b8da640 1 -- 172.21.15.78:0/3248817840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3644108ab0 con 0x7f3644102980 2024-04-25T01:45:06.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.026+0000 7f363a7fc640 1 -- 172.21.15.78:0/3248817840 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3634042030 con 0x7f3644102980 2024-04-25T01:45:06.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.027+0000 7f363a7fc640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36300a2b90 0x7f36300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:06.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.027+0000 7f3649676640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36300a2b90 0x7f36300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:06.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.027+0000 7f363a7fc640 1 -- 172.21.15.78:0/3248817840 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f36340efe00 con 0x7f3644102980 2024-04-25T01:45:06.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.027+0000 7f3649676640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36300a2b90 0x7f36300a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3640005fd0 tx=0x7f3640005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:06.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.030+0000 7f363a7fc640 1 -- 172.21.15.78:0/3248817840 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f36340bca20 con 0x7f3644102980 2024-04-25T01:45:06.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.243+0000 7f364b8da640 1 -- 172.21.15.78:0/3248817840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f364402d170 con 0x7f3644102980 2024-04-25T01:45:06.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.244+0000 7f363a7fc640 1 -- 172.21.15.78:0/3248817840 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f36340bc320 con 0x7f3644102980 2024-04-25T01:45:06.237 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:06.237 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:43:17.355192+0000","modified":"2024-04-25T01:43:44.435289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24455,"mds_2":24448,"mds_3":24392,"mds_4":24379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1337519666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1337519666},{"type":"v1","addr":"172.21.15.78:6839","nonce":1337519666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1963410683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1963410683},{"type":"v1","addr":"172.21.15.78:6841","nonce":1963410683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24379":{"gid":24379,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1969344843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1969344843},{"type":"v1","addr":"172.21.15.152:6839","nonce":1969344843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24379,24491,24455,24392,24448]},"id":1}]} 2024-04-25T01:45:06.237 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-04-25T01:45:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.246+0000 7f364b8da640 1 -- 172.21.15.78:0/3248817840 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36300a2b90 msgr2=0x7f36300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.246+0000 7f364b8da640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36300a2b90 0x7f36300a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3640005fd0 tx=0x7f3640005850 comp rx=0 tx=0).stop 2024-04-25T01:45:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.246+0000 7f364b8da640 1 -- 172.21.15.78:0/3248817840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3644102980 msgr2=0x7f3644197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.246+0000 7f364b8da640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3644102980 0x7f3644197fd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3634009230 tx=0x7f3634004360 comp rx=0 tx=0).stop 2024-04-25T01:45:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.246+0000 7f364b8da640 1 -- 172.21.15.78:0/3248817840 shutdown_connections 2024-04-25T01:45:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.246+0000 7f364b8da640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3644103340 0x7f364419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.246+0000 7f364b8da640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f36300a2b90 0x7f36300a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.246+0000 7f364b8da640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3644102980 0x7f3644197fd0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.246+0000 7f364b8da640 1 --2- 172.21.15.78:0/3248817840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3644101790 0x7f3644197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.246+0000 7f364b8da640 1 -- 172.21.15.78:0/3248817840 >> 172.21.15.78:0/3248817840 conn(0x7f36440fcd20 msgr2=0x7f3644105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.246+0000 7f364b8da640 1 -- 172.21.15.78:0/3248817840 shutdown_connections 2024-04-25T01:45:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.246+0000 7f364b8da640 1 -- 172.21.15.78:0/3248817840 wait complete. 2024-04-25T01:45:06.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:06 smithi078 ceph-mon[29469]: pgmap v346: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:45:06.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:06 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2267612632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:45:06.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:06 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2919674270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:45:06.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:06 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1584901684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:45:06.249 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:06 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3868783345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:45:06.250 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:45:06.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:06 smithi152 ceph-mon[32465]: pgmap v346: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:45:06.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:06 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2267612632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:45:06.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:06 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2919674270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:45:06.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:06 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1584901684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:45:06.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:06 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3868783345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:45:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.358+0000 7f67fc912640 1 -- 172.21.15.78:0/973626327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f8101870 msgr2=0x7f67f8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.358+0000 7f67fc912640 1 --2- 172.21.15.78:0/973626327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f8101870 0x7f67f8101cf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009900 tx=0x7f67e802a410 comp rx=0 tx=0).stop 2024-04-25T01:45:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.358+0000 7f67fc912640 1 -- 172.21.15.78:0/973626327 shutdown_connections 2024-04-25T01:45:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.358+0000 7f67fc912640 1 --2- 172.21.15.78:0/973626327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67f8102230 0x7f67f8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.358+0000 7f67fc912640 1 --2- 172.21.15.78:0/973626327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f8101870 0x7f67f8101cf0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.358+0000 7f67fc912640 1 --2- 172.21.15.78:0/973626327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67f8100680 0x7f67f8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.358+0000 7f67fc912640 1 -- 172.21.15.78:0/973626327 >> 172.21.15.78:0/973626327 conn(0x7f67f80fbbf0 msgr2=0x7f67f80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.358+0000 7f67fc912640 1 -- 172.21.15.78:0/973626327 shutdown_connections 2024-04-25T01:45:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.358+0000 7f67fc912640 1 -- 172.21.15.78:0/973626327 wait complete. 2024-04-25T01:45:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.358+0000 7f67fc912640 1 Processor -- start 2024-04-25T01:45:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67fc912640 1 -- start start 2024-04-25T01:45:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67fc912640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67f8100680 0x7f67f8196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67fc912640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f8101870 0x7f67f8196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67fc912640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67f8102230 0x7f67f819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67fc912640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67f810c2e0 con 0x7f67f8101870 2024-04-25T01:45:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67fc912640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f67f810c160 con 0x7f67f8100680 2024-04-25T01:45:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67fc912640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67f810c460 con 0x7f67f8102230 2024-04-25T01:45:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67f5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f8101870 0x7f67f8196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67f5d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f8101870 0x7f67f8196e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40952/0 (socket says 172.21.15.78:40952) 2024-04-25T01:45:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67f5d9b640 1 -- 172.21.15.78:0/2097597276 learned_addr learned my addr 172.21.15.78:0/2097597276 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67f659c640 1 --2- 172.21.15.78:0/2097597276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67f8100680 0x7f67f8196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67f6d9d640 1 --2- 172.21.15.78:0/2097597276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67f8102230 0x7f67f819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67f5d9b640 1 -- 172.21.15.78:0/2097597276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67f8102230 msgr2=0x7f67f819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67f5d9b640 1 --2- 172.21.15.78:0/2097597276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67f8102230 0x7f67f819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67f5d9b640 1 -- 172.21.15.78:0/2097597276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67f8100680 msgr2=0x7f67f8196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67f5d9b640 1 --2- 172.21.15.78:0/2097597276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67f8100680 0x7f67f8196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67f5d9b640 1 -- 172.21.15.78:0/2097597276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f819e540 con 0x7f67f8101870 2024-04-25T01:45:06.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67f5d9b640 1 --2- 172.21.15.78:0/2097597276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f8101870 0x7f67f8196e60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009230 tx=0x7f67e8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:06.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.359+0000 7f67e77fe640 1 -- 172.21.15.78:0/2097597276 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e803f030 con 0x7f67f8101870 2024-04-25T01:45:06.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.360+0000 7f67fc912640 1 -- 172.21.15.78:0/2097597276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f819e7d0 con 0x7f67f8101870 2024-04-25T01:45:06.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.360+0000 7f67e77fe640 1 -- 172.21.15.78:0/2097597276 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e8003ce0 con 0x7f67f8101870 2024-04-25T01:45:06.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.360+0000 7f67e77fe640 1 -- 172.21.15.78:0/2097597276 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8035070 con 0x7f67f8101870 2024-04-25T01:45:06.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.360+0000 7f67fc912640 1 -- 172.21.15.78:0/2097597276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f819ed10 con 0x7f67f8101870 2024-04-25T01:45:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.361+0000 7f67fc912640 1 -- 172.21.15.78:0/2097597276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f8107940 con 0x7f67f8101870 2024-04-25T01:45:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.362+0000 7f67e77fe640 1 -- 172.21.15.78:0/2097597276 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f67e8042030 con 0x7f67f8101870 2024-04-25T01:45:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.362+0000 7f67e77fe640 1 --2- 172.21.15.78:0/2097597276 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f67e00a2b90 0x7f67e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.362+0000 7f67f659c640 1 --2- 172.21.15.78:0/2097597276 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f67e00a2b90 0x7f67e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.363+0000 7f67e77fe640 1 -- 172.21.15.78:0/2097597276 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f67e80efd70 con 0x7f67f8101870 2024-04-25T01:45:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.363+0000 7f67f659c640 1 --2- 172.21.15.78:0/2097597276 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f67e00a2b90 0x7f67e00a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f67f81016d0 tx=0x7f67ec005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:06.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.370+0000 7f67e77fe640 1 -- 172.21.15.78:0/2097597276 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f67e80bc990 con 0x7f67f8101870 2024-04-25T01:45:06.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:06 smithi196 ceph-mon[32978]: pgmap v346: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:45:06.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:06 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2267612632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:45:06.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:06 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2919674270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:45:06.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:06 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1584901684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:45:06.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:06 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3868783345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:45:06.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.555+0000 7f67fc912640 1 -- 172.21.15.78:0/2097597276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67f802d2f0 con 0x7f67f8101870 2024-04-25T01:45:06.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.557+0000 7f67e77fe640 1 -- 172.21.15.78:0/2097597276 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24592 (secure 0 0 0) 0x7f67e80bc290 con 0x7f67f8101870 2024-04-25T01:45:06.550 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:06.550 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:44:18.500577+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:45:06.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.559+0000 7f67fc912640 1 -- 172.21.15.78:0/2097597276 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f67e00a2b90 msgr2=0x7f67e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.559+0000 7f67fc912640 1 --2- 172.21.15.78:0/2097597276 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f67e00a2b90 0x7f67e00a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f67f81016d0 tx=0x7f67ec005f90 comp rx=0 tx=0).stop 2024-04-25T01:45:06.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.559+0000 7f67fc912640 1 -- 172.21.15.78:0/2097597276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f8101870 msgr2=0x7f67f8196e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.559+0000 7f67fc912640 1 --2- 172.21.15.78:0/2097597276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f8101870 0x7f67f8196e60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009230 tx=0x7f67e8004360 comp rx=0 tx=0).stop 2024-04-25T01:45:06.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.559+0000 7f67fc912640 1 -- 172.21.15.78:0/2097597276 shutdown_connections 2024-04-25T01:45:06.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.559+0000 7f67fc912640 1 --2- 172.21.15.78:0/2097597276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67f8102230 0x7f67f819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.559+0000 7f67fc912640 1 --2- 172.21.15.78:0/2097597276 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f67e00a2b90 0x7f67e00a5050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.559+0000 7f67fc912640 1 --2- 172.21.15.78:0/2097597276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67f8101870 0x7f67f8196e60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.559+0000 7f67fc912640 1 --2- 172.21.15.78:0/2097597276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67f8100680 0x7f67f8196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.559+0000 7f67fc912640 1 -- 172.21.15.78:0/2097597276 >> 172.21.15.78:0/2097597276 conn(0x7f67f80fbbf0 msgr2=0x7f67f8104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:06.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.559+0000 7f67fc912640 1 -- 172.21.15.78:0/2097597276 shutdown_connections 2024-04-25T01:45:06.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.559+0000 7f67fc912640 1 -- 172.21.15.78:0/2097597276 wait complete. 2024-04-25T01:45:06.564 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:45:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 -- 172.21.15.78:0/3335528939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac01009c0 msgr2=0x7f0ac0100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 --2- 172.21.15.78:0/3335528939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac01009c0 0x7f0ac0100e20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009900 tx=0x7f0ab002a410 comp rx=0 tx=0).stop 2024-04-25T01:45:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 -- 172.21.15.78:0/3335528939 shutdown_connections 2024-04-25T01:45:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 --2- 172.21.15.78:0/3335528939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ac0101360 0x7f0ac01017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 --2- 172.21.15.78:0/3335528939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac01009c0 0x7f0ac0100e20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 --2- 172.21.15.78:0/3335528939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ac0100060 0x7f0ac0100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 -- 172.21.15.78:0/3335528939 >> 172.21.15.78:0/3335528939 conn(0x7f0ac00fbc20 msgr2=0x7f0ac00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 -- 172.21.15.78:0/3335528939 shutdown_connections 2024-04-25T01:45:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 -- 172.21.15.78:0/3335528939 wait complete. 2024-04-25T01:45:06.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 Processor -- start 2024-04-25T01:45:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 -- start start 2024-04-25T01:45:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ac0100060 0x7f0ac0199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac01009c0 0x7f0ac019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ac0101360 0x7f0ac01a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac010f5f0 con 0x7f0ac01009c0 2024-04-25T01:45:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac010f470 con 0x7f0ac0101360 2024-04-25T01:45:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.673+0000 7f0ac727d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac010f770 con 0x7f0ac0100060 2024-04-25T01:45:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ac4818640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac01009c0 0x7f0ac019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ac4818640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac01009c0 0x7f0ac019a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40972/0 (socket says 172.21.15.78:40972) 2024-04-25T01:45:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ac4818640 1 -- 172.21.15.78:0/1190279703 learned_addr learned my addr 172.21.15.78:0/1190279703 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ac5019640 1 --2- 172.21.15.78:0/1190279703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ac0100060 0x7f0ac0199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ac4818640 1 -- 172.21.15.78:0/1190279703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ac0100060 msgr2=0x7f0ac0199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ac4818640 1 --2- 172.21.15.78:0/1190279703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ac0100060 0x7f0ac0199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ac4818640 1 -- 172.21.15.78:0/1190279703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ac0101360 msgr2=0x7f0ac01a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ac4818640 1 --2- 172.21.15.78:0/1190279703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ac0101360 0x7f0ac01a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ac4818640 1 -- 172.21.15.78:0/1190279703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac01a18e0 con 0x7f0ac01009c0 2024-04-25T01:45:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ac4818640 1 --2- 172.21.15.78:0/1190279703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac01009c0 0x7f0ac019a1e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009230 tx=0x7f0ab0004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ab67fc640 1 -- 172.21.15.78:0/1190279703 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab003f030 con 0x7f0ac01009c0 2024-04-25T01:45:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ac727d640 1 -- 172.21.15.78:0/1190279703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac01a1b70 con 0x7f0ac01009c0 2024-04-25T01:45:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ab67fc640 1 -- 172.21.15.78:0/1190279703 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ab0003ce0 con 0x7f0ac01009c0 2024-04-25T01:45:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ab67fc640 1 -- 172.21.15.78:0/1190279703 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0035070 con 0x7f0ac01009c0 2024-04-25T01:45:06.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.674+0000 7f0ac727d640 1 -- 172.21.15.78:0/1190279703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac01a20b0 con 0x7f0ac01009c0 2024-04-25T01:45:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.675+0000 7f0ac727d640 1 -- 172.21.15.78:0/1190279703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac010af30 con 0x7f0ac01009c0 2024-04-25T01:45:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.677+0000 7f0ab67fc640 1 -- 172.21.15.78:0/1190279703 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0ab0042030 con 0x7f0ac01009c0 2024-04-25T01:45:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.677+0000 7f0ab67fc640 1 --2- 172.21.15.78:0/1190279703 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0aac0a2b90 0x7f0aac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.678+0000 7f0ab67fc640 1 -- 172.21.15.78:0/1190279703 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f0ab00efe30 con 0x7f0ac01009c0 2024-04-25T01:45:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.678+0000 7f0ac5019640 1 --2- 172.21.15.78:0/1190279703 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0aac0a2b90 0x7f0aac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.678+0000 7f0ac5019640 1 --2- 172.21.15.78:0/1190279703 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0aac0a2b90 0x7f0aac0a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0abc005fd0 tx=0x7f0abc005e20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.681+0000 7f0ab67fc640 1 -- 172.21.15.78:0/1190279703 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0ab00bcb70 con 0x7f0ac01009c0 2024-04-25T01:45:06.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.906+0000 7f0ac727d640 1 -- 172.21.15.78:0/1190279703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ac002d320 con 0x7f0ac01009c0 2024-04-25T01:45:06.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.907+0000 7f0ab67fc640 1 -- 172.21.15.78:0/1190279703 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f0ab00bc470 con 0x7f0ac01009c0 2024-04-25T01:45:06.900 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:06.900 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:43:17.355192+0000","modified":"2024-04-25T01:43:44.435289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24455,"mds_2":24448,"mds_3":24392,"mds_4":24379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1337519666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1337519666},{"type":"v1","addr":"172.21.15.78:6839","nonce":1337519666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1963410683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1963410683},{"type":"v1","addr":"172.21.15.78:6841","nonce":1963410683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24379":{"gid":24379,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1969344843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1969344843},{"type":"v1","addr":"172.21.15.152:6839","nonce":1969344843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24379,24491,24455,24392,24448]},"id":1}]} 2024-04-25T01:45:06.900 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-04-25T01:45:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.909+0000 7f0ac727d640 1 -- 172.21.15.78:0/1190279703 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0aac0a2b90 msgr2=0x7f0aac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.909+0000 7f0ac727d640 1 --2- 172.21.15.78:0/1190279703 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0aac0a2b90 0x7f0aac0a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0abc005fd0 tx=0x7f0abc005e20 comp rx=0 tx=0).stop 2024-04-25T01:45:06.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.909+0000 7f0ac727d640 1 -- 172.21.15.78:0/1190279703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac01009c0 msgr2=0x7f0ac019a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.909+0000 7f0ac727d640 1 --2- 172.21.15.78:0/1190279703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac01009c0 0x7f0ac019a1e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009230 tx=0x7f0ab0004360 comp rx=0 tx=0).stop 2024-04-25T01:45:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.909+0000 7f0ac727d640 1 -- 172.21.15.78:0/1190279703 shutdown_connections 2024-04-25T01:45:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.909+0000 7f0ac727d640 1 --2- 172.21.15.78:0/1190279703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ac0101360 0x7f0ac01a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.909+0000 7f0ac727d640 1 --2- 172.21.15.78:0/1190279703 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f0aac0a2b90 0x7f0aac0a5050 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.909+0000 7f0ac727d640 1 --2- 172.21.15.78:0/1190279703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ac01009c0 0x7f0ac019a1e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.909+0000 7f0ac727d640 1 --2- 172.21.15.78:0/1190279703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ac0100060 0x7f0ac0199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.909+0000 7f0ac727d640 1 -- 172.21.15.78:0/1190279703 >> 172.21.15.78:0/1190279703 conn(0x7f0ac00fbc20 msgr2=0x7f0ac00fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.909+0000 7f0ac727d640 1 -- 172.21.15.78:0/1190279703 shutdown_connections 2024-04-25T01:45:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:06.909+0000 7f0ac727d640 1 -- 172.21.15.78:0/1190279703 wait complete. 2024-04-25T01:45:06.914 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T01:43:17.355192+0000', 'modified': '2024-04-25T01:43:44.435289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24491, 'mds_1': 24455, 'mds_2': 24448, 'mds_3': 24392, 'mds_4': 24379}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14634': {'gid': 14634, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6839/1337519666', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 1337519666}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 1337519666}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14646': {'gid': 14646, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6841/1963410683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6840', 'nonce': 1963410683}, {'type': 'v1', 'addr': '172.21.15.78:6841', 'nonce': 1963410683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24379': {'gid': 24379, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.196:6833/4110091360', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 4110091360}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 4110091360}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24392': {'gid': 24392, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.152:6833/923579368', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 923579368}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 923579368}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/2927651724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 2927651724}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 2927651724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6835/4251210610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6834', 'nonce': 4251210610}, {'type': 'v1', 'addr': '172.21.15.152:6835', 'nonce': 4251210610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6837/1117133390', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 1117133390}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 1117133390}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.196:6839/460829167', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 460829167}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 460829167}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.152:6837/4112159743', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6836', 'nonce': 4112159743}, {'type': 'v1', 'addr': '172.21.15.152:6837', 'nonce': 4112159743}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6839/1969344843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 1969344843}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 1969344843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24491, 'qdb_cluster': [24379, 24491, 24455, 24392, 24448]} 2024-04-25T01:45:06.914 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T01:45:06.914 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-25T01:45:07.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.023+0000 7f1b29fd5640 1 -- 172.21.15.78:0/3147362831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b24104b30 msgr2=0x7f1b24106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.023+0000 7f1b29fd5640 1 --2- 172.21.15.78:0/3147362831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b24104b30 0x7f1b24106f20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009900 tx=0x7f1b1802a410 comp rx=0 tx=0).stop 2024-04-25T01:45:07.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.023+0000 7f1b29fd5640 1 -- 172.21.15.78:0/3147362831 shutdown_connections 2024-04-25T01:45:07.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.023+0000 7f1b29fd5640 1 --2- 172.21.15.78:0/3147362831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b24107460 0x7f1b241098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.023+0000 7f1b29fd5640 1 --2- 172.21.15.78:0/3147362831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b24104b30 0x7f1b24106f20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.023+0000 7f1b29fd5640 1 --2- 172.21.15.78:0/3147362831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b24102200 0x7f1b241045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.023+0000 7f1b29fd5640 1 -- 172.21.15.78:0/3147362831 >> 172.21.15.78:0/3147362831 conn(0x7f1b240fbbf0 msgr2=0x7f1b240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.023+0000 7f1b29fd5640 1 -- 172.21.15.78:0/3147362831 shutdown_connections 2024-04-25T01:45:07.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.023+0000 7f1b29fd5640 1 -- 172.21.15.78:0/3147362831 wait complete. 2024-04-25T01:45:07.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.023+0000 7f1b29fd5640 1 Processor -- start 2024-04-25T01:45:07.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.023+0000 7f1b29fd5640 1 -- start start 2024-04-25T01:45:07.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.023+0000 7f1b29fd5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b24102200 0x7f1b24197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b29fd5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b24104b30 0x7f1b241980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b29fd5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b24107460 0x7f1b2419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b29fd5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2410d470 con 0x7f1b24104b30 2024-04-25T01:45:07.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b29fd5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2410d2f0 con 0x7f1b24102200 2024-04-25T01:45:07.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b29fd5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2410d5f0 con 0x7f1b24107460 2024-04-25T01:45:07.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b22ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b24104b30 0x7f1b241980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b22ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b24104b30 0x7f1b241980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40984/0 (socket says 172.21.15.78:40984) 2024-04-25T01:45:07.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b22ffd640 1 -- 172.21.15.78:0/3636097103 learned_addr learned my addr 172.21.15.78:0/3636097103 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b23fff640 1 --2- 172.21.15.78:0/3636097103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b24107460 0x7f1b2419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b237fe640 1 --2- 172.21.15.78:0/3636097103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b24102200 0x7f1b24197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b22ffd640 1 -- 172.21.15.78:0/3636097103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b24107460 msgr2=0x7f1b2419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b22ffd640 1 --2- 172.21.15.78:0/3636097103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b24107460 0x7f1b2419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b22ffd640 1 -- 172.21.15.78:0/3636097103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b24102200 msgr2=0x7f1b24197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b22ffd640 1 --2- 172.21.15.78:0/3636097103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b24102200 0x7f1b24197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b22ffd640 1 -- 172.21.15.78:0/3636097103 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2419f790 con 0x7f1b24104b30 2024-04-25T01:45:07.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b22ffd640 1 --2- 172.21.15.78:0/3636097103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b24104b30 0x7f1b241980b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1b18031610 tx=0x7f1b18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b20ff9640 1 -- 172.21.15.78:0/3636097103 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1803f030 con 0x7f1b24104b30 2024-04-25T01:45:07.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b20ff9640 1 -- 172.21.15.78:0/3636097103 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b18003ad0 con 0x7f1b24104b30 2024-04-25T01:45:07.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b29fd5640 1 -- 172.21.15.78:0/3636097103 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2419fa20 con 0x7f1b24104b30 2024-04-25T01:45:07.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b20ff9640 1 -- 172.21.15.78:0/3636097103 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18035070 con 0x7f1b24104b30 2024-04-25T01:45:07.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.024+0000 7f1b29fd5640 1 -- 172.21.15.78:0/3636097103 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2419ff90 con 0x7f1b24104b30 2024-04-25T01:45:07.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.026+0000 7f1b20ff9640 1 -- 172.21.15.78:0/3636097103 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1b18042030 con 0x7f1b24104b30 2024-04-25T01:45:07.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b2cdb7740 1 Processor -- start 2024-04-25T01:45:07.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b2cdb7740 1 -- start start 2024-04-25T01:45:07.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b20ff9640 1 --2- 172.21.15.78:0/3636097103 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1b0c0a2b90 0x7f1b0c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b2cdb7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 0x55d4d8e0f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b2cdb7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e14930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b2cdb7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e17430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b2cdb7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d4d8d72f00 con 0x55d4d8e0fe30 2024-04-25T01:45:07.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b2cdb7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d4d8d1b750 con 0x55d4d8e14fa0 2024-04-25T01:45:07.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b2cdb7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d4d8d6eec0 con 0x55d4d8e0f440 2024-04-25T01:45:07.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b20ff9640 1 -- 172.21.15.78:0/3636097103 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f1b180efb60 con 0x7f1b24104b30 2024-04-25T01:45:07.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b23fff640 1 --2- 172.21.15.78:0/3636097103 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1b0c0a2b90 0x7f1b0c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b22ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e14930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b22ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e14930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40996/0 (socket says 172.21.15.78:40996) 2024-04-25T01:45:07.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b22ffd640 1 -- 172.21.15.78:0/1384259535 learned_addr learned my addr 172.21.15.78:0/1384259535 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.027+0000 7f1b237fe640 1 --2- 172.21.15.78:0/1384259535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 0x55d4d8e0f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.028+0000 7f1b22ffd640 1 -- 172.21.15.78:0/1384259535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 msgr2=0x55d4d8e0f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.028+0000 7f1b23fff640 1 --2- 172.21.15.78:0/1384259535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e17430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.028+0000 7f1b22ffd640 1 --2- 172.21.15.78:0/1384259535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 0x55d4d8e0f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.028+0000 7f1b22ffd640 1 -- 172.21.15.78:0/1384259535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 msgr2=0x55d4d8e17430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.028+0000 7f1b22ffd640 1 --2- 172.21.15.78:0/1384259535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e17430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.028+0000 7f1b22ffd640 1 -- 172.21.15.78:0/1384259535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4d8a872d0 con 0x55d4d8e0fe30 2024-04-25T01:45:07.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.028+0000 7f1b23fff640 1 --2- 172.21.15.78:0/1384259535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e17430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:07.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.028+0000 7f1b23fff640 1 --2- 172.21.15.78:0/3636097103 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1b0c0a2b90 0x7f1b0c0a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009eb0 tx=0x7f1b1c00c7f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.028+0000 7f1b22ffd640 1 --2- 172.21.15.78:0/1384259535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e14930 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1b180bbd80 tx=0x7f1b180bbaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d345473d857e797a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.028+0000 7f1b09ffb640 1 -- 172.21.15.78:0/1384259535 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18049040 con 0x55d4d8e0fe30 2024-04-25T01:45:07.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.028+0000 7f1b09ffb640 1 -- 172.21.15.78:0/1384259535 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b180c4cf0 con 0x55d4d8e0fe30 2024-04-25T01:45:07.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.028+0000 7f1b09ffb640 1 -- 172.21.15.78:0/1384259535 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b180c0080 con 0x55d4d8e0fe30 2024-04-25T01:45:07.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.029+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/1384259535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0fe30 msgr2=0x55d4d8e14930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.029+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/1384259535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e14930 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1b180bbd80 tx=0x7f1b180bbaa0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.029+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/1384259535 shutdown_connections 2024-04-25T01:45:07.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.029+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/1384259535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e17430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.029+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/1384259535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e14930 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.029+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/1384259535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 0x55d4d8e0f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.029+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/1384259535 >> 172.21.15.78:0/1384259535 conn(0x55d4d8e0aab0 msgr2=0x55d4d8e0aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.029+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/1384259535 shutdown_connections 2024-04-25T01:45:07.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.029+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/1384259535 wait complete. 2024-04-25T01:45:07.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.029+0000 7f1b2cdb7740 1 -- start start 2024-04-25T01:45:07.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.029+0000 7f1b2cdb7740 1 Processor -- start 2024-04-25T01:45:07.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.029+0000 7f1b2cdb7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 0x55d4d8e19610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b2cdb7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e19b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b2cdb7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e20b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b2cdb7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d4d8e17d10 con 0x55d4d8e14fa0 2024-04-25T01:45:07.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b2cdb7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d4d8d2ecf0 con 0x55d4d8e0fe30 2024-04-25T01:45:07.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b2cdb7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d4d8e17e90 con 0x55d4d8e0f440 2024-04-25T01:45:07.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b2cdb7740 10 client.0 _ll_register_callbacks cb 0x55d4d8d2e6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:07.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b23fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e20b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b23fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e20b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41000/0 (socket says 172.21.15.78:41000) 2024-04-25T01:45:07.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b23fff640 1 -- 172.21.15.78:0/2860223613 learned_addr learned my addr 172.21.15.78:0/2860223613 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b22ffd640 1 --2- 172.21.15.78:0/2860223613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e19b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b237fe640 1 --2- 172.21.15.78:0/2860223613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 0x55d4d8e19610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b23fff640 1 -- 172.21.15.78:0/2860223613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 msgr2=0x55d4d8e19610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b23fff640 1 --2- 172.21.15.78:0/2860223613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 0x55d4d8e19610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b23fff640 1 -- 172.21.15.78:0/2860223613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e0fe30 msgr2=0x55d4d8e19b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b23fff640 1 --2- 172.21.15.78:0/2860223613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e19b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b23fff640 1 -- 172.21.15.78:0/2860223613 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4d8b516b0 con 0x55d4d8e14fa0 2024-04-25T01:45:07.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b237fe640 1 --2- 172.21.15.78:0/2860223613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 0x55d4d8e19610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:07.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b23fff640 1 --2- 172.21.15.78:0/2860223613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e20b90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c01aee0 tx=0x7f1b1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66c5723abeb64b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:07.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b02ffd640 1 -- 172.21.15.78:0/2860223613 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c018e60 con 0x55d4d8e14fa0 2024-04-25T01:45:07.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b02ffd640 1 -- 172.21.15.78:0/2860223613 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f1b1c019400 con 0x55d4d8e14fa0 2024-04-25T01:45:07.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/2860223613 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b180c4cf0 con 0x55d4d8e14fa0 2024-04-25T01:45:07.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b02ffd640 1 -- 172.21.15.78:0/2860223613 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1b1c019600 con 0x55d4d8e14fa0 2024-04-25T01:45:07.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b2cdb7740 10 client.15240 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:07.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.030+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/2860223613 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d4d8e23b50 con 0x55d4d8e14fa0 2024-04-25T01:45:07.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.031+0000 7f1b02ffd640 1 -- 172.21.15.78:0/2860223613 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b180c4cf0 con 0x55d4d8e14fa0 2024-04-25T01:45:07.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.031+0000 7f1b02ffd640 1 -- 172.21.15.78:0/2860223613 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1b1c024960 con 0x55d4d8e14fa0 2024-04-25T01:45:07.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.031+0000 7f1b2cdb7740 10 client.15240 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:07.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.031+0000 7f1b2cdb7740 10 client.15240 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-25T01:45:07.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.031+0000 7f1b2cdb7740 20 client.15240 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:07.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.031+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/2860223613 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x55d4d8e26250 0x55d4d8e276c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.031+0000 7f1b2cdb7740 4 client.15240 mds_command: new command op to 24379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:45:07.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.031+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/2860223613 --> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4d8d80440 con 0x55d4d8e26250 2024-04-25T01:45:07.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.031+0000 7f1b237fe640 1 --2- 172.21.15.78:0/2860223613 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x55d4d8e26250 0x55d4d8e276c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.032+0000 7f1b237fe640 1 --2- 172.21.15.78:0/2860223613 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x55d4d8e26250 0x55d4d8e276c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e54b0457aaed4f1 server_cookie=5b2d42e384512462 in_seq=0 out_seq=0 2024-04-25T01:45:07.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.032+0000 7f1b02ffd640 10 client.15240 ms_handle_connect on v2:172.21.15.196:6832/4110091360 2024-04-25T01:45:07.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.032+0000 7f1b02ffd640 1 -- 172.21.15.78:0/2860223613 <== mds.4 v2:172.21.15.196:6832/4110091360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4d8d80440 con 0x55d4d8e26250 2024-04-25T01:45:07.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.032+0000 7f1b02ffd640 10 client.15240 handle_command_reply: tid=0 2024-04-25T01:45:07.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.032+0000 7f1b2cdb7740 1 client.15240 shutdown 2024-04-25T01:45:07.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.033+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/2860223613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e14fa0 msgr2=0x55d4d8e20b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.033+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/2860223613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e20b90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c01aee0 tx=0x7f1b1c004a40 comp rx=0 tx=0).stop 2024-04-25T01:45:07.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.033+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/2860223613 shutdown_connections 2024-04-25T01:45:07.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.033+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/2860223613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e20b90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.033+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/2860223613 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x55d4d8e26250 0x55d4d8e276c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.033+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/2860223613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e19b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.033+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/2860223613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 0x55d4d8e19610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.033+0000 7f1b02ffd640 0 client.15240 ms_handle_reset on v2:172.21.15.196:6832/4110091360 2024-04-25T01:45:07.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.033+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/2860223613 >> 172.21.15.78:0/2860223613 conn(0x55d4d8e0aab0 msgr2=0x55d4d8e09ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.033+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/2860223613 shutdown_connections 2024-04-25T01:45:07.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.033+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/2860223613 wait complete. 2024-04-25T01:45:07.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.033+0000 7f1b2cdb7740 20 client.15240 trim_cache size 0 max 16384 2024-04-25T01:45:07.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.050+0000 7f1b2cdb7740 1 Processor -- start 2024-04-25T01:45:07.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b2cdb7740 1 -- start start 2024-04-25T01:45:07.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b2cdb7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 0x55d4d8e0a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b2cdb7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e05690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b2cdb7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e19220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b2cdb7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d4d8e25a00 con 0x55d4d8e0fe30 2024-04-25T01:45:07.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b2cdb7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d4d8e260d0 con 0x55d4d8e14fa0 2024-04-25T01:45:07.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b2cdb7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d4d8e267a0 con 0x55d4d8e0f440 2024-04-25T01:45:07.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b22ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e05690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b22ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e05690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41016/0 (socket says 172.21.15.78:41016) 2024-04-25T01:45:07.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b22ffd640 1 -- 172.21.15.78:0/2398900500 learned_addr learned my addr 172.21.15.78:0/2398900500 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b22ffd640 1 -- 172.21.15.78:0/2398900500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 msgr2=0x55d4d8e0a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b237fe640 1 --2- 172.21.15.78:0/2398900500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 0x55d4d8e0a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b22ffd640 1 --2- 172.21.15.78:0/2398900500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 0x55d4d8e0a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b22ffd640 1 -- 172.21.15.78:0/2398900500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 msgr2=0x55d4d8e19220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b23fff640 1 --2- 172.21.15.78:0/2398900500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e19220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b22ffd640 1 --2- 172.21.15.78:0/2398900500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e19220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b22ffd640 1 -- 172.21.15.78:0/2398900500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4d8e106c0 con 0x55d4d8e0fe30 2024-04-25T01:45:07.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b237fe640 1 --2- 172.21.15.78:0/2398900500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 0x55d4d8e0a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:07.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b22ffd640 1 --2- 172.21.15.78:0/2398900500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e05690 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1b180ed690 tx=0x7f1b18005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdda4b4a35a40508 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b097fa640 1 -- 172.21.15.78:0/2398900500 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18049040 con 0x55d4d8e0fe30 2024-04-25T01:45:07.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b097fa640 1 -- 172.21.15.78:0/2398900500 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b180ba120 con 0x55d4d8e0fe30 2024-04-25T01:45:07.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.051+0000 7f1b097fa640 1 -- 172.21.15.78:0/2398900500 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1803b460 con 0x55d4d8e0fe30 2024-04-25T01:45:07.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.052+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/2398900500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0fe30 msgr2=0x55d4d8e05690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.052+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/2398900500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e05690 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1b180ed690 tx=0x7f1b18005c00 comp rx=0 tx=0).stop 2024-04-25T01:45:07.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.052+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/2398900500 shutdown_connections 2024-04-25T01:45:07.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.052+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/2398900500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e19220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.052+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/2398900500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e05690 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.052+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/2398900500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0f440 0x55d4d8e0a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.052+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/2398900500 >> 172.21.15.78:0/2398900500 conn(0x55d4d8e0aab0 msgr2=0x55d4d8e11290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.052+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/2398900500 shutdown_connections 2024-04-25T01:45:07.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.052+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/2398900500 wait complete. 2024-04-25T01:45:07.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.052+0000 7f1b2cdb7740 1 -- start start 2024-04-25T01:45:07.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b2cdb7740 1 Processor -- start 2024-04-25T01:45:07.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b2cdb7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0f440 0x55d4d8e8a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b2cdb7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e8c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b2cdb7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e8ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b2cdb7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d4d8e27540 con 0x55d4d8e0f440 2024-04-25T01:45:07.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b2cdb7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d4d8e26e70 con 0x55d4d8e14fa0 2024-04-25T01:45:07.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b2cdb7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d4d8e27c10 con 0x55d4d8e0fe30 2024-04-25T01:45:07.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b2cdb7740 10 client.0 _ll_register_callbacks cb 0x7f1b1c01abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:07.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0f440 0x55d4d8e8a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0f440 0x55d4d8e8a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41028/0 (socket says 172.21.15.78:41028) 2024-04-25T01:45:07.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b237fe640 1 -- 172.21.15.78:0/1514224359 learned_addr learned my addr 172.21.15.78:0/1514224359 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b22ffd640 1 --2- 172.21.15.78:0/1514224359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e8c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b23fff640 1 --2- 172.21.15.78:0/1514224359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e8ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b237fe640 1 -- 172.21.15.78:0/1514224359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0fe30 msgr2=0x55d4d8e8c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b237fe640 1 --2- 172.21.15.78:0/1514224359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e8c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b237fe640 1 -- 172.21.15.78:0/1514224359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 msgr2=0x55d4d8e8ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b237fe640 1 --2- 172.21.15.78:0/1514224359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e8ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b237fe640 1 -- 172.21.15.78:0/1514224359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4d8e8f1d0 con 0x55d4d8e0f440 2024-04-25T01:45:07.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b237fe640 1 --2- 172.21.15.78:0/1514224359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0f440 0x55d4d8e8a1f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1b14004700 tx=0x7f1b14004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0c9daf5a59d676 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:07.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b02ffd640 1 -- 172.21.15.78:0/1514224359 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1401e420 con 0x55d4d8e0f440 2024-04-25T01:45:07.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.053+0000 7f1b02ffd640 1 -- 172.21.15.78:0/1514224359 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f1b1401e5c0 con 0x55d4d8e0f440 2024-04-25T01:45:07.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.054+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/1514224359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b180ba120 con 0x55d4d8e0f440 2024-04-25T01:45:07.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.054+0000 7f1b02ffd640 1 -- 172.21.15.78:0/1514224359 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1b1402c050 con 0x55d4d8e0f440 2024-04-25T01:45:07.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.054+0000 7f1b2cdb7740 10 client.15246 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:07.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.054+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/1514224359 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d4d8e90490 con 0x55d4d8e0f440 2024-04-25T01:45:07.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.054+0000 7f1b02ffd640 1 -- 172.21.15.78:0/1514224359 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b180ba120 con 0x55d4d8e0f440 2024-04-25T01:45:07.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.054+0000 7f1b02ffd640 1 -- 172.21.15.78:0/1514224359 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1b14012450 con 0x55d4d8e0f440 2024-04-25T01:45:07.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.054+0000 7f1b2cdb7740 10 client.15246 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:07.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.054+0000 7f1b2cdb7740 10 client.15246 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-25T01:45:07.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.054+0000 7f1b2cdb7740 20 client.15246 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:07.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.054+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/1514224359 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x55d4d8e950e0 0x55d4d8e91520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.054+0000 7f1b2cdb7740 4 client.15246 mds_command: new command op to 24379 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:45:07.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.054+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/1514224359 --> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] -- command(tid 0: {"prefix": "status"}) -- 0x55d4d8d7a790 con 0x55d4d8e950e0 2024-04-25T01:45:07.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.054+0000 7f1b22ffd640 1 --2- 172.21.15.78:0/1514224359 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x55d4d8e950e0 0x55d4d8e91520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b02ffd640 10 client.15246 ms_handle_connect on v2:172.21.15.196:6832/4110091360 2024-04-25T01:45:07.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b22ffd640 1 --2- 172.21.15.78:0/1514224359 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x55d4d8e950e0 0x55d4d8e91520 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=287aa3cf7547354d server_cookie=d9432e842e3caf26 in_seq=0 out_seq=0 2024-04-25T01:45:07.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b02ffd640 1 -- 172.21.15.78:0/1514224359 <== mds.4 v2:172.21.15.196:6832/4110091360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d4d8d7a790 con 0x55d4d8e950e0 2024-04-25T01:45:07.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b02ffd640 10 client.15246 handle_command_reply: tid=0 2024-04-25T01:45:07.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b2cdb7740 1 client.15246 shutdown 2024-04-25T01:45:07.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/1514224359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0f440 msgr2=0x55d4d8e8a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/1514224359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0f440 0x55d4d8e8a1f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1b14004700 tx=0x7f1b14004410 comp rx=0 tx=0).stop 2024-04-25T01:45:07.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/1514224359 shutdown_connections 2024-04-25T01:45:07.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/1514224359 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x55d4d8e950e0 0x55d4d8e91520 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b02ffd640 0 client.15246 ms_handle_reset on v2:172.21.15.196:6832/4110091360 2024-04-25T01:45:07.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/1514224359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4d8e14fa0 0x55d4d8e8ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/1514224359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4d8e0fe30 0x55d4d8e8c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b2cdb7740 1 --2- 172.21.15.78:0/1514224359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d4d8e0f440 0x55d4d8e8a1f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/1514224359 >> 172.21.15.78:0/1514224359 conn(0x55d4d8e0aab0 msgr2=0x55d4d8e19220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/1514224359 shutdown_connections 2024-04-25T01:45:07.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b2cdb7740 1 -- 172.21.15.78:0/1514224359 wait complete. 2024-04-25T01:45:07.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.055+0000 7f1b2cdb7740 20 client.15246 trim_cache size 0 max 16384 2024-04-25T01:45:07.056 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:45:07.056 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "df36bea8-02a3-11ef-bc93-c7b262605968", 2024-04-25T01:45:07.056 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 4, 2024-04-25T01:45:07.056 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24379, 2024-04-25T01:45:07.056 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T01:45:07.056 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T01:45:07.056 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T01:45:07.057 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 83.623757088000005, 2024-04-25T01:45:07.057 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-04-25T01:45:07.057 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2024-04-25T01:45:07.057 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T01:45:07.057 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 146.92623341300001 2024-04-25T01:45:07.057 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:45:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.056+0000 7f1b29fd5640 1 -- 172.21.15.78:0/3636097103 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1b0c0a2b90 msgr2=0x7f1b0c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.056+0000 7f1b29fd5640 1 --2- 172.21.15.78:0/3636097103 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1b0c0a2b90 0x7f1b0c0a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009eb0 tx=0x7f1b1c00c7f0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.056+0000 7f1b29fd5640 1 -- 172.21.15.78:0/3636097103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b24104b30 msgr2=0x7f1b241980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.056+0000 7f1b29fd5640 1 --2- 172.21.15.78:0/3636097103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b24104b30 0x7f1b241980b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1b18031610 tx=0x7f1b18004270 comp rx=0 tx=0).stop 2024-04-25T01:45:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.056+0000 7f1b29fd5640 1 -- 172.21.15.78:0/3636097103 shutdown_connections 2024-04-25T01:45:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.056+0000 7f1b29fd5640 1 --2- 172.21.15.78:0/3636097103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b24107460 0x7f1b2419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.056+0000 7f1b29fd5640 1 --2- 172.21.15.78:0/3636097103 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f1b0c0a2b90 0x7f1b0c0a5050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.056+0000 7f1b29fd5640 1 --2- 172.21.15.78:0/3636097103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b24104b30 0x7f1b241980b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.056+0000 7f1b29fd5640 1 --2- 172.21.15.78:0/3636097103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b24102200 0x7f1b24197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.056+0000 7f1b29fd5640 1 -- 172.21.15.78:0/3636097103 >> 172.21.15.78:0/3636097103 conn(0x7f1b240fbbf0 msgr2=0x7f1b24102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.056+0000 7f1b29fd5640 1 -- 172.21.15.78:0/3636097103 shutdown_connections 2024-04-25T01:45:07.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.056+0000 7f1b29fd5640 1 -- 172.21.15.78:0/3636097103 wait complete. 2024-04-25T01:45:07.059 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-25T01:45:07.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.166+0000 7fd26d321640 1 -- 172.21.15.78:0/1000325235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268102200 msgr2=0x7fd2681045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.166+0000 7fd26d321640 1 --2- 172.21.15.78:0/1000325235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268102200 0x7fd2681045f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd258009900 tx=0x7fd25802a410 comp rx=0 tx=0).stop 2024-04-25T01:45:07.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.166+0000 7fd26d321640 1 -- 172.21.15.78:0/1000325235 shutdown_connections 2024-04-25T01:45:07.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.166+0000 7fd26d321640 1 --2- 172.21.15.78:0/1000325235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd268107460 0x7fd2681098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.166+0000 7fd26d321640 1 --2- 172.21.15.78:0/1000325235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd268104b30 0x7fd268106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.166+0000 7fd26d321640 1 --2- 172.21.15.78:0/1000325235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268102200 0x7fd2681045f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.166+0000 7fd26d321640 1 -- 172.21.15.78:0/1000325235 >> 172.21.15.78:0/1000325235 conn(0x7fd2680fbbf0 msgr2=0x7fd2680fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.166+0000 7fd26d321640 1 -- 172.21.15.78:0/1000325235 shutdown_connections 2024-04-25T01:45:07.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.166+0000 7fd26d321640 1 -- 172.21.15.78:0/1000325235 wait complete. 2024-04-25T01:45:07.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd26d321640 1 Processor -- start 2024-04-25T01:45:07.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd26d321640 1 -- start start 2024-04-25T01:45:07.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd26d321640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd268102200 0x7fd268197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd26d321640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268104b30 0x7fd268198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd26d321640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd268107460 0x7fd26819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd26d321640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd26810d420 con 0x7fd268104b30 2024-04-25T01:45:07.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd26d321640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd26810d2a0 con 0x7fd268107460 2024-04-25T01:45:07.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd26d321640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd26810d5a0 con 0x7fd268102200 2024-04-25T01:45:07.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd25ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268104b30 0x7fd268198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd25ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268104b30 0x7fd268198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41048/0 (socket says 172.21.15.78:41048) 2024-04-25T01:45:07.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd25ffff640 1 -- 172.21.15.78:0/4157408315 learned_addr learned my addr 172.21.15.78:0/4157408315 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd2677fe640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd268107460 0x7fd26819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd25ffff640 1 -- 172.21.15.78:0/4157408315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd268102200 msgr2=0x7fd268197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd266ffd640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd268102200 0x7fd268197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd25ffff640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd268102200 0x7fd268197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.167+0000 7fd25ffff640 1 -- 172.21.15.78:0/4157408315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd268107460 msgr2=0x7fd26819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.168+0000 7fd25ffff640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd268107460 0x7fd26819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.168+0000 7fd25ffff640 1 -- 172.21.15.78:0/4157408315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26819f710 con 0x7fd268104b30 2024-04-25T01:45:07.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.168+0000 7fd2677fe640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd268107460 0x7fd26819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:07.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.168+0000 7fd25ffff640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268104b30 0x7fd268198080 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd250005fd0 tx=0x7fd25000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.168+0000 7fd264ff9640 1 -- 172.21.15.78:0/4157408315 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250017070 con 0x7fd268104b30 2024-04-25T01:45:07.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.168+0000 7fd26d321640 1 -- 172.21.15.78:0/4157408315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26819fa00 con 0x7fd268104b30 2024-04-25T01:45:07.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.168+0000 7fd264ff9640 1 -- 172.21.15.78:0/4157408315 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd250009c30 con 0x7fd268104b30 2024-04-25T01:45:07.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.168+0000 7fd264ff9640 1 -- 172.21.15.78:0/4157408315 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250010460 con 0x7fd268104b30 2024-04-25T01:45:07.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.168+0000 7fd26d321640 1 -- 172.21.15.78:0/4157408315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26819ffa0 con 0x7fd268104b30 2024-04-25T01:45:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.170+0000 7fd264ff9640 1 -- 172.21.15.78:0/4157408315 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd250003e70 con 0x7fd268104b30 2024-04-25T01:45:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.170+0000 7fd27002a740 1 Processor -- start 2024-04-25T01:45:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.170+0000 7fd27002a740 1 -- start start 2024-04-25T01:45:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd27002a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce45440 0x55a96ce45860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd27002a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45e30 0x55a96ce4a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd27002a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce4afa0 0x55a96ce4d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd27002a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a96cda8f00 con 0x55a96ce45440 2024-04-25T01:45:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd27002a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a96cd51750 con 0x55a96ce4afa0 2024-04-25T01:45:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd27002a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a96cda4ec0 con 0x55a96ce45e30 2024-04-25T01:45:07.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd2677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce45440 0x55a96ce45860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd264ff9640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd2540a2b90 0x7fd2540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd2677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce45440 0x55a96ce45860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41056/0 (socket says 172.21.15.78:41056) 2024-04-25T01:45:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd2677fe640 1 -- 172.21.15.78:0/513730640 learned_addr learned my addr 172.21.15.78:0/513730640 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd266ffd640 1 --2- 172.21.15.78:0/513730640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45e30 0x55a96ce4a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd25ffff640 1 --2- 172.21.15.78:0/513730640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce4afa0 0x55a96ce4d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd2677fe640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd2540a2b90 0x7fd2540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd264ff9640 1 -- 172.21.15.78:0/4157408315 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fd2500cf140 con 0x7fd268104b30 2024-04-25T01:45:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd2677fe640 1 -- 172.21.15.78:0/513730640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45e30 msgr2=0x55a96ce4a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd2677fe640 1 --2- 172.21.15.78:0/513730640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45e30 0x55a96ce4a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd2677fe640 1 -- 172.21.15.78:0/513730640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce4afa0 msgr2=0x55a96ce4d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd2677fe640 1 --2- 172.21.15.78:0/513730640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce4afa0 0x55a96ce4d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd2677fe640 1 -- 172.21.15.78:0/513730640 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a96cabd2d0 con 0x55a96ce45440 2024-04-25T01:45:07.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd266ffd640 1 --2- 172.21.15.78:0/513730640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45e30 0x55a96ce4a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:45:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.171+0000 7fd25ffff640 1 --2- 172.21.15.78:0/513730640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce4afa0 0x55a96ce4d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.172+0000 7fd2677fe640 1 --2- 172.21.15.78:0/513730640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce45440 0x55a96ce45860 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd260012920 tx=0x7fd260012de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b40001ba090c6e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.172+0000 7fd25d7fa640 1 -- 172.21.15.78:0/513730640 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260022070 con 0x55a96ce45440 2024-04-25T01:45:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.172+0000 7fd25d7fa640 1 -- 172.21.15.78:0/513730640 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd260012030 con 0x55a96ce45440 2024-04-25T01:45:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.172+0000 7fd25d7fa640 1 -- 172.21.15.78:0/513730640 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260005260 con 0x55a96ce45440 2024-04-25T01:45:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.172+0000 7fd27002a740 1 -- 172.21.15.78:0/513730640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce45440 msgr2=0x55a96ce45860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.172+0000 7fd2677fe640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd2540a2b90 0x7fd2540a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd260020000 tx=0x7fd26001f000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.172+0000 7fd27002a740 1 --2- 172.21.15.78:0/513730640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce45440 0x55a96ce45860 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd260012920 tx=0x7fd260012de0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 -- 172.21.15.78:0/513730640 shutdown_connections 2024-04-25T01:45:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 --2- 172.21.15.78:0/513730640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce4afa0 0x55a96ce4d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 --2- 172.21.15.78:0/513730640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45e30 0x55a96ce4a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 --2- 172.21.15.78:0/513730640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce45440 0x55a96ce45860 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 -- 172.21.15.78:0/513730640 >> 172.21.15.78:0/513730640 conn(0x55a96ce40ab0 msgr2=0x55a96ce40eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 -- 172.21.15.78:0/513730640 shutdown_connections 2024-04-25T01:45:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 -- 172.21.15.78:0/513730640 wait complete. 2024-04-25T01:45:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 -- start start 2024-04-25T01:45:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 Processor -- start 2024-04-25T01:45:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45440 0x55a96ce4f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce45e30 0x55a96ce4fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce4afa0 0x55a96ce56dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a96ce4df50 con 0x55a96ce45e30 2024-04-25T01:45:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a96cd64cf0 con 0x55a96ce45440 2024-04-25T01:45:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a96ce4e0d0 con 0x55a96ce4afa0 2024-04-25T01:45:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd27002a740 10 client.0 _ll_register_callbacks cb 0x55a96cd646e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd25ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce45e30 0x55a96ce4fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd25ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce45e30 0x55a96ce4fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41072/0 (socket says 172.21.15.78:41072) 2024-04-25T01:45:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd25ffff640 1 -- 172.21.15.78:0/4205229816 learned_addr learned my addr 172.21.15.78:0/4205229816 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.173+0000 7fd266ffd640 1 --2- 172.21.15.78:0/4205229816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45440 0x55a96ce4f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd2677fe640 1 --2- 172.21.15.78:0/4205229816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce4afa0 0x55a96ce56dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd25ffff640 1 -- 172.21.15.78:0/4205229816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce4afa0 msgr2=0x55a96ce56dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd25ffff640 1 --2- 172.21.15.78:0/4205229816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce4afa0 0x55a96ce56dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd25ffff640 1 -- 172.21.15.78:0/4205229816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45440 msgr2=0x55a96ce4f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd25ffff640 1 --2- 172.21.15.78:0/4205229816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45440 0x55a96ce4f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd25ffff640 1 -- 172.21.15.78:0/4205229816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a96cb876b0 con 0x55a96ce45e30 2024-04-25T01:45:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd2677fe640 1 --2- 172.21.15.78:0/4205229816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce4afa0 0x55a96ce56dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd25ffff640 1 --2- 172.21.15.78:0/4205229816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce45e30 0x55a96ce4fdf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd250097ac0 tx=0x7fd25006a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cd45d67614b7a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd24e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd24e7fc640 1 -- 172.21.15.78:0/4205229816 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250086080 con 0x55a96ce45e30 2024-04-25T01:45:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd24e7fc640 1 -- 172.21.15.78:0/4205229816 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fd250097cd0 con 0x55a96ce45e30 2024-04-25T01:45:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd27002a740 1 -- 172.21.15.78:0/4205229816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd260012030 con 0x55a96ce45e30 2024-04-25T01:45:07.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd24e7fc640 1 -- 172.21.15.78:0/4205229816 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd250066740 con 0x55a96ce45e30 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd27002a740 10 client.15258 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd27002a740 1 -- 172.21.15.78:0/4205229816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a96ce59dd0 con 0x55a96ce45e30 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd24e7fc640 1 -- 172.21.15.78:0/4205229816 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd260012030 con 0x55a96ce45e30 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd24e7fc640 1 -- 172.21.15.78:0/4205229816 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd25006a160 con 0x55a96ce45e30 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.174+0000 7fd27002a740 10 client.15258 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.175+0000 7fd27002a740 10 client.15258 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.175+0000 7fd27002a740 20 client.15258 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.175+0000 7fd27002a740 1 --2- 172.21.15.78:0/4205229816 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x55a96ce5c3b0 0x55a96ce5d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.175+0000 7fd27002a740 4 client.15258 mds_command: new command op to 24392 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.175+0000 7fd27002a740 1 -- 172.21.15.78:0/4205229816 --> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a96cdb6440 con 0x55a96ce5c3b0 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.175+0000 7fd266ffd640 1 --2- 172.21.15.78:0/4205229816 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x55a96ce5c3b0 0x55a96ce5d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.175+0000 7fd24e7fc640 10 client.15258 ms_handle_connect on v2:172.21.15.152:6832/923579368 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.175+0000 7fd266ffd640 1 --2- 172.21.15.78:0/4205229816 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x55a96ce5c3b0 0x55a96ce5d820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=85b4c63227af3525 server_cookie=d95d6defa44706d4 in_seq=0 out_seq=0 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd24e7fc640 1 -- 172.21.15.78:0/4205229816 <== mds.3 v2:172.21.15.152:6832/923579368 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a96cdb6440 con 0x55a96ce5c3b0 2024-04-25T01:45:07.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd24e7fc640 10 client.15258 handle_command_reply: tid=0 2024-04-25T01:45:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd27002a740 1 client.15258 shutdown 2024-04-25T01:45:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd27002a740 1 -- 172.21.15.78:0/4205229816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce45e30 msgr2=0x55a96ce4fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd27002a740 1 --2- 172.21.15.78:0/4205229816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce45e30 0x55a96ce4fdf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd250097ac0 tx=0x7fd25006a5e0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd27002a740 1 -- 172.21.15.78:0/4205229816 shutdown_connections 2024-04-25T01:45:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd27002a740 1 --2- 172.21.15.78:0/4205229816 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x55a96ce5c3b0 0x55a96ce5d820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd27002a740 1 --2- 172.21.15.78:0/4205229816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce4afa0 0x55a96ce56dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd27002a740 1 --2- 172.21.15.78:0/4205229816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce45e30 0x55a96ce4fdf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd24e7fc640 0 client.15258 ms_handle_reset on v2:172.21.15.152:6832/923579368 2024-04-25T01:45:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd27002a740 1 --2- 172.21.15.78:0/4205229816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45440 0x55a96ce4f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd27002a740 1 -- 172.21.15.78:0/4205229816 >> 172.21.15.78:0/4205229816 conn(0x55a96ce40ab0 msgr2=0x55a96ce3fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd27002a740 1 -- 172.21.15.78:0/4205229816 shutdown_connections 2024-04-25T01:45:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd27002a740 1 -- 172.21.15.78:0/4205229816 wait complete. 2024-04-25T01:45:07.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.176+0000 7fd27002a740 20 client.15258 trim_cache size 0 max 16384 2024-04-25T01:45:07.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.194+0000 7fd27002a740 1 Processor -- start 2024-04-25T01:45:07.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd27002a740 1 -- start start 2024-04-25T01:45:07.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd27002a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45440 0x55a96ce4d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd27002a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45e30 0x55a96ce56d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd27002a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce4afa0 0x55a96ce472d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd27002a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a96ce5bb60 con 0x55a96ce4afa0 2024-04-25T01:45:07.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd27002a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a96ce5c230 con 0x55a96ce45440 2024-04-25T01:45:07.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd27002a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a96ce5c900 con 0x55a96ce45e30 2024-04-25T01:45:07.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd2677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce4afa0 0x55a96ce472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd2677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce4afa0 0x55a96ce472d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41084/0 (socket says 172.21.15.78:41084) 2024-04-25T01:45:07.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd2677fe640 1 -- 172.21.15.78:0/2052084766 learned_addr learned my addr 172.21.15.78:0/2052084766 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd2677fe640 1 -- 172.21.15.78:0/2052084766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45e30 msgr2=0x55a96ce56d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd25ffff640 1 --2- 172.21.15.78:0/2052084766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45e30 0x55a96ce56d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd2677fe640 1 --2- 172.21.15.78:0/2052084766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45e30 0x55a96ce56d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd2677fe640 1 -- 172.21.15.78:0/2052084766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45440 msgr2=0x55a96ce4d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd2677fe640 1 --2- 172.21.15.78:0/2052084766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45440 0x55a96ce4d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd2677fe640 1 -- 172.21.15.78:0/2052084766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a96ce408e0 con 0x55a96ce4afa0 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.195+0000 7fd2677fe640 1 --2- 172.21.15.78:0/2052084766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce4afa0 0x55a96ce472d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd260010870 tx=0x7fd260004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed63bfb8aa0b2b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.196+0000 7fd25cff9640 1 -- 172.21.15.78:0/2052084766 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260036070 con 0x55a96ce4afa0 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.196+0000 7fd25cff9640 1 -- 172.21.15.78:0/2052084766 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd260005a60 con 0x55a96ce4afa0 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.196+0000 7fd25cff9640 1 -- 172.21.15.78:0/2052084766 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260022040 con 0x55a96ce4afa0 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.196+0000 7fd27002a740 1 -- 172.21.15.78:0/2052084766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce4afa0 msgr2=0x55a96ce472d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.196+0000 7fd27002a740 1 --2- 172.21.15.78:0/2052084766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce4afa0 0x55a96ce472d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd260010870 tx=0x7fd260004a00 comp rx=0 tx=0).stop 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.196+0000 7fd27002a740 1 -- 172.21.15.78:0/2052084766 shutdown_connections 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.196+0000 7fd27002a740 1 --2- 172.21.15.78:0/2052084766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce4afa0 0x55a96ce472d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.196+0000 7fd27002a740 1 --2- 172.21.15.78:0/2052084766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45e30 0x55a96ce56d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.196+0000 7fd27002a740 1 --2- 172.21.15.78:0/2052084766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45440 0x55a96ce4d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.196+0000 7fd27002a740 1 -- 172.21.15.78:0/2052084766 >> 172.21.15.78:0/2052084766 conn(0x55a96ce40ab0 msgr2=0x55a96ce3b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.196+0000 7fd27002a740 1 -- 172.21.15.78:0/2052084766 shutdown_connections 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.196+0000 7fd27002a740 1 -- 172.21.15.78:0/2052084766 wait complete. 2024-04-25T01:45:07.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd27002a740 1 -- start start 2024-04-25T01:45:07.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd27002a740 1 Processor -- start 2024-04-25T01:45:07.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd27002a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45440 0x55a96ce49d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd27002a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45e30 0x55a96ce56d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd27002a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce4afa0 0x55a96cec03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd27002a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a96ce5d6a0 con 0x55a96ce4afa0 2024-04-25T01:45:07.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd27002a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a96ce5cfd0 con 0x55a96ce45e30 2024-04-25T01:45:07.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd27002a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a96ce5dd70 con 0x55a96ce45440 2024-04-25T01:45:07.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd27002a740 10 client.0 _ll_register_callbacks cb 0x7fd250003630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:07.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd2677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce4afa0 0x55a96cec03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd2677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce4afa0 0x55a96cec03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41096/0 (socket says 172.21.15.78:41096) 2024-04-25T01:45:07.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd2677fe640 1 -- 172.21.15.78:0/2590885039 learned_addr learned my addr 172.21.15.78:0/2590885039 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd25ffff640 1 --2- 172.21.15.78:0/2590885039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45e30 0x55a96ce56d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd266ffd640 1 --2- 172.21.15.78:0/2590885039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45440 0x55a96ce49d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd2677fe640 1 -- 172.21.15.78:0/2590885039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45440 msgr2=0x55a96ce49d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd2677fe640 1 --2- 172.21.15.78:0/2590885039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45440 0x55a96ce49d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd2677fe640 1 -- 172.21.15.78:0/2590885039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45e30 msgr2=0x55a96ce56d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd2677fe640 1 --2- 172.21.15.78:0/2590885039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45e30 0x55a96ce56d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd2677fe640 1 -- 172.21.15.78:0/2590885039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a96cec0be0 con 0x55a96ce4afa0 2024-04-25T01:45:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd25ffff640 1 --2- 172.21.15.78:0/2590885039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45e30 0x55a96ce56d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd2677fe640 1 --2- 172.21.15.78:0/2590885039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce4afa0 0x55a96cec03b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd26000a620 tx=0x7fd260005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bcb4d1f7178bfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.197+0000 7fd24e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd24e7fc640 1 -- 172.21.15.78:0/2590885039 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260022050 con 0x55a96ce4afa0 2024-04-25T01:45:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd24e7fc640 1 -- 172.21.15.78:0/2590885039 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fd260005580 con 0x55a96ce4afa0 2024-04-25T01:45:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd27002a740 1 -- 172.21.15.78:0/2590885039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd260005a60 con 0x55a96ce4afa0 2024-04-25T01:45:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd24e7fc640 1 -- 172.21.15.78:0/2590885039 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd260036040 con 0x55a96ce4afa0 2024-04-25T01:45:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd27002a740 10 client.15264 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd27002a740 1 -- 172.21.15.78:0/2590885039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a96cec1ee0 con 0x55a96ce4afa0 2024-04-25T01:45:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd24e7fc640 1 -- 172.21.15.78:0/2590885039 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd260005a60 con 0x55a96ce4afa0 2024-04-25T01:45:07.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd24e7fc640 1 -- 172.21.15.78:0/2590885039 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd26003acd0 con 0x55a96ce4afa0 2024-04-25T01:45:07.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd27002a740 10 client.15264 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:07.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd27002a740 10 client.15264 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-25T01:45:07.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd27002a740 20 client.15264 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:07.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd27002a740 1 --2- 172.21.15.78:0/2590885039 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x55a96cec63d0 0x55a96cec50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd27002a740 4 client.15264 mds_command: new command op to 24392 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:45:07.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd27002a740 1 -- 172.21.15.78:0/2590885039 --> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] -- command(tid 0: {"prefix": "status"}) -- 0x55a96cdb0790 con 0x55a96cec63d0 2024-04-25T01:45:07.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.198+0000 7fd266ffd640 1 --2- 172.21.15.78:0/2590885039 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x55a96cec63d0 0x55a96cec50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd24e7fc640 10 client.15264 ms_handle_connect on v2:172.21.15.152:6832/923579368 2024-04-25T01:45:07.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd266ffd640 1 --2- 172.21.15.78:0/2590885039 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x55a96cec63d0 0x55a96cec50e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f0ade6fc0f7b81db server_cookie=a473cc3a4afd6ca4 in_seq=0 out_seq=0 2024-04-25T01:45:07.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd24e7fc640 1 -- 172.21.15.78:0/2590885039 <== mds.3 v2:172.21.15.152:6832/923579368 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a96cdb0790 con 0x55a96cec63d0 2024-04-25T01:45:07.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd24e7fc640 10 client.15264 handle_command_reply: tid=0 2024-04-25T01:45:07.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd27002a740 1 client.15264 shutdown 2024-04-25T01:45:07.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd27002a740 1 -- 172.21.15.78:0/2590885039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce4afa0 msgr2=0x55a96cec03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd27002a740 1 --2- 172.21.15.78:0/2590885039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce4afa0 0x55a96cec03b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd26000a620 tx=0x7fd260005c80 comp rx=0 tx=0).stop 2024-04-25T01:45:07.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd27002a740 1 -- 172.21.15.78:0/2590885039 shutdown_connections 2024-04-25T01:45:07.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd27002a740 1 --2- 172.21.15.78:0/2590885039 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x55a96cec63d0 0x55a96cec50e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd27002a740 1 --2- 172.21.15.78:0/2590885039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a96ce4afa0 0x55a96cec03b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd27002a740 1 --2- 172.21.15.78:0/2590885039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a96ce45e30 0x55a96ce56d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd24e7fc640 0 client.15264 ms_handle_reset on v2:172.21.15.152:6832/923579368 2024-04-25T01:45:07.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd27002a740 1 --2- 172.21.15.78:0/2590885039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a96ce45440 0x55a96ce49d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd27002a740 1 -- 172.21.15.78:0/2590885039 >> 172.21.15.78:0/2590885039 conn(0x55a96ce40ab0 msgr2=0x55a96ce472d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd27002a740 1 -- 172.21.15.78:0/2590885039 shutdown_connections 2024-04-25T01:45:07.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd27002a740 1 -- 172.21.15.78:0/2590885039 wait complete. 2024-04-25T01:45:07.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.199+0000 7fd27002a740 20 client.15264 trim_cache size 0 max 16384 2024-04-25T01:45:07.200 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:45:07.200 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "df36bea8-02a3-11ef-bc93-c7b262605968", 2024-04-25T01:45:07.200 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 3, 2024-04-25T01:45:07.200 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24392, 2024-04-25T01:45:07.200 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T01:45:07.200 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T01:45:07.200 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T01:45:07.200 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 84.784094292000006, 2024-04-25T01:45:07.200 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-04-25T01:45:07.200 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2024-04-25T01:45:07.200 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T01:45:07.201 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 144.199267733 2024-04-25T01:45:07.201 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:45:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.200+0000 7fd26d321640 1 -- 172.21.15.78:0/4157408315 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd2540a2b90 msgr2=0x7fd2540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.200+0000 7fd26d321640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd2540a2b90 0x7fd2540a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd260020000 tx=0x7fd26001f000 comp rx=0 tx=0).stop 2024-04-25T01:45:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.200+0000 7fd26d321640 1 -- 172.21.15.78:0/4157408315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268104b30 msgr2=0x7fd268198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.200+0000 7fd26d321640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268104b30 0x7fd268198080 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd250005fd0 tx=0x7fd25000dde0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.200+0000 7fd26d321640 1 -- 172.21.15.78:0/4157408315 shutdown_connections 2024-04-25T01:45:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.200+0000 7fd26d321640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd268107460 0x7fd26819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.200+0000 7fd26d321640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd2540a2b90 0x7fd2540a5050 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.200+0000 7fd26d321640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd268104b30 0x7fd268198080 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.200+0000 7fd26d321640 1 --2- 172.21.15.78:0/4157408315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd268102200 0x7fd268197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.200+0000 7fd26d321640 1 -- 172.21.15.78:0/4157408315 >> 172.21.15.78:0/4157408315 conn(0x7fd2680fbbf0 msgr2=0x7fd268102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.200+0000 7fd26d321640 1 -- 172.21.15.78:0/4157408315 shutdown_connections 2024-04-25T01:45:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.200+0000 7fd26d321640 1 -- 172.21.15.78:0/4157408315 wait complete. 2024-04-25T01:45:07.203 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:07 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3248817840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:07.203 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:07 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2097597276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:07.203 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:07 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1190279703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:07.204 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T01:45:07.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:07 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3248817840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:07.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:07 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2097597276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:07.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:07 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1190279703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:07.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 -- 172.21.15.78:0/1550832941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784100680 msgr2=0x7f8784100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 --2- 172.21.15.78:0/1550832941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784100680 0x7f8784100aa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8780009900 tx=0x7f878002a410 comp rx=0 tx=0).stop 2024-04-25T01:45:07.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 -- 172.21.15.78:0/1550832941 shutdown_connections 2024-04-25T01:45:07.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 --2- 172.21.15.78:0/1550832941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8784102230 0x7f8784108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 --2- 172.21.15.78:0/1550832941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8784101870 0x7f8784101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 --2- 172.21.15.78:0/1550832941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784100680 0x7f8784100aa0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 -- 172.21.15.78:0/1550832941 >> 172.21.15.78:0/1550832941 conn(0x7f87840fbbf0 msgr2=0x7f87840fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 -- 172.21.15.78:0/1550832941 shutdown_connections 2024-04-25T01:45:07.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 -- 172.21.15.78:0/1550832941 wait complete. 2024-04-25T01:45:07.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 Processor -- start 2024-04-25T01:45:07.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 -- start start 2024-04-25T01:45:07.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8784100680 0x7f8784197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784101870 0x7f8784198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8784102230 0x7f878419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f878410c350 con 0x7f8784101870 2024-04-25T01:45:07.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f878410c1d0 con 0x7f8784100680 2024-04-25T01:45:07.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f878b529640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f878410c4d0 con 0x7f8784102230 2024-04-25T01:45:07.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.311+0000 7f8788ac4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784101870 0x7f8784198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.312+0000 7f8788ac4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784101870 0x7f8784198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41122/0 (socket says 172.21.15.78:41122) 2024-04-25T01:45:07.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.312+0000 7f8788ac4640 1 -- 172.21.15.78:0/1596261995 learned_addr learned my addr 172.21.15.78:0/1596261995 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.312+0000 7f87892c5640 1 --2- 172.21.15.78:0/1596261995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8784100680 0x7f8784197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.312+0000 7f8788ac4640 1 -- 172.21.15.78:0/1596261995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8784102230 msgr2=0x7f878419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.312+0000 7f8788ac4640 1 --2- 172.21.15.78:0/1596261995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8784102230 0x7f878419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.312+0000 7f8788ac4640 1 -- 172.21.15.78:0/1596261995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8784100680 msgr2=0x7f8784197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.312+0000 7f8788ac4640 1 --2- 172.21.15.78:0/1596261995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8784100680 0x7f8784197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.312+0000 7f8788ac4640 1 -- 172.21.15.78:0/1596261995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878419f740 con 0x7f8784101870 2024-04-25T01:45:07.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.312+0000 7f8788ac4640 1 --2- 172.21.15.78:0/1596261995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784101870 0x7f8784198060 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f877400d950 tx=0x7f877400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.312+0000 7f877a7fc640 1 -- 172.21.15.78:0/1596261995 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8774017070 con 0x7f8784101870 2024-04-25T01:45:07.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.312+0000 7f877a7fc640 1 -- 172.21.15.78:0/1596261995 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8774009c30 con 0x7f8784101870 2024-04-25T01:45:07.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.312+0000 7f878b529640 1 -- 172.21.15.78:0/1596261995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878419fa30 con 0x7f8784101870 2024-04-25T01:45:07.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.312+0000 7f877a7fc640 1 -- 172.21.15.78:0/1596261995 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877400f6b0 con 0x7f8784101870 2024-04-25T01:45:07.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.312+0000 7f878b529640 1 -- 172.21.15.78:0/1596261995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87841a0000 con 0x7f8784101870 2024-04-25T01:45:07.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.314+0000 7f877a7fc640 1 -- 172.21.15.78:0/1596261995 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87740109d0 con 0x7f8784101870 2024-04-25T01:45:07.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.315+0000 7f877a7fc640 1 --2- 172.21.15.78:0/1596261995 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f87700a2b90 0x7f87700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.315+0000 7f87892c5640 1 --2- 172.21.15.78:0/1596261995 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f87700a2b90 0x7f87700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.315+0000 7f877a7fc640 1 -- 172.21.15.78:0/1596261995 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f87740cf010 con 0x7f8784101870 2024-04-25T01:45:07.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.315+0000 7f87892c5640 1 --2- 172.21.15.78:0/1596261995 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f87700a2b90 0x7f87700a5050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8780005ab0 tx=0x7f8780000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f878e23a740 1 Processor -- start 2024-04-25T01:45:07.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f878e23a740 1 -- start start 2024-04-25T01:45:07.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f878e23a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3440 0x55e5c53b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f878e23a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3e30 0x55e5c53b8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f878e23a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b8fa0 0x55e5c53bb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f878e23a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e5c5316f00 con 0x55e5c53b3440 2024-04-25T01:45:07.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f878e23a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55e5c52bf750 con 0x55e5c53b3e30 2024-04-25T01:45:07.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f878e23a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e5c5312ec0 con 0x55e5c53b8fa0 2024-04-25T01:45:07.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f87892c5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3440 0x55e5c53b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f87892c5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3440 0x55e5c53b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41136/0 (socket says 172.21.15.78:41136) 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f87892c5640 1 -- 172.21.15.78:0/576391161 learned_addr learned my addr 172.21.15.78:0/576391161 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f8789ac6640 1 --2- 172.21.15.78:0/576391161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b8fa0 0x55e5c53bb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f87892c5640 1 -- 172.21.15.78:0/576391161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b8fa0 msgr2=0x55e5c53bb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f87892c5640 1 --2- 172.21.15.78:0/576391161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b8fa0 0x55e5c53bb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f87892c5640 1 -- 172.21.15.78:0/576391161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3e30 msgr2=0x55e5c53b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f87892c5640 1 --2- 172.21.15.78:0/576391161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3e30 0x55e5c53b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f87892c5640 1 -- 172.21.15.78:0/576391161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c502b2d0 con 0x55e5c53b3440 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.316+0000 7f87892c5640 1 --2- 172.21.15.78:0/576391161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3440 0x55e5c53b3860 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f87841016d0 tx=0x7f878003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7919f11347de6e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.317+0000 7f876f7fe640 1 -- 172.21.15.78:0/576391161 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780035040 con 0x55e5c53b3440 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.317+0000 7f876f7fe640 1 -- 172.21.15.78:0/576391161 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f878004d070 con 0x55e5c53b3440 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.317+0000 7f876f7fe640 1 -- 172.21.15.78:0/576391161 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87800049e0 con 0x55e5c53b3440 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.317+0000 7f878e23a740 1 -- 172.21.15.78:0/576391161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3440 msgr2=0x55e5c53b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.317+0000 7f878e23a740 1 --2- 172.21.15.78:0/576391161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3440 0x55e5c53b3860 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f87841016d0 tx=0x7f878003d000 comp rx=0 tx=0).stop 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.317+0000 7f878e23a740 1 -- 172.21.15.78:0/576391161 shutdown_connections 2024-04-25T01:45:07.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.317+0000 7f878e23a740 1 --2- 172.21.15.78:0/576391161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b8fa0 0x55e5c53bb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.317+0000 7f878e23a740 1 --2- 172.21.15.78:0/576391161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3e30 0x55e5c53b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.317+0000 7f878e23a740 1 --2- 172.21.15.78:0/576391161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3440 0x55e5c53b3860 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.317+0000 7f878e23a740 1 -- 172.21.15.78:0/576391161 >> 172.21.15.78:0/576391161 conn(0x55e5c53aeab0 msgr2=0x55e5c53aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.317+0000 7f878e23a740 1 -- 172.21.15.78:0/576391161 shutdown_connections 2024-04-25T01:45:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.317+0000 7f878e23a740 1 -- 172.21.15.78:0/576391161 wait complete. 2024-04-25T01:45:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.318+0000 7f878e23a740 1 -- start start 2024-04-25T01:45:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.318+0000 7f878e23a740 1 Processor -- start 2024-04-25T01:45:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.318+0000 7f878e23a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b3440 0x55e5c53bd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.318+0000 7f878e23a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3e30 0x55e5c53bdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.318+0000 7f878e23a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b8fa0 0x55e5c53c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.318+0000 7f878e23a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e5c53bbef0 con 0x55e5c53b3e30 2024-04-25T01:45:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.318+0000 7f878e23a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55e5c52d2cf0 con 0x55e5c53b8fa0 2024-04-25T01:45:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.318+0000 7f878e23a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e5c53bc070 con 0x55e5c53b3440 2024-04-25T01:45:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.318+0000 7f878e23a740 10 client.0 _ll_register_callbacks cb 0x55e5c52d26e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:07.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.318+0000 7f8788ac4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3e30 0x55e5c53bdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.318+0000 7f8788ac4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3e30 0x55e5c53bdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41140/0 (socket says 172.21.15.78:41140) 2024-04-25T01:45:07.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.318+0000 7f8788ac4640 1 -- 172.21.15.78:0/728640691 learned_addr learned my addr 172.21.15.78:0/728640691 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.318+0000 7f8789ac6640 1 --2- 172.21.15.78:0/728640691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b8fa0 0x55e5c53c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.318+0000 7f87892c5640 1 --2- 172.21.15.78:0/728640691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b3440 0x55e5c53bd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f8788ac4640 1 -- 172.21.15.78:0/728640691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b3440 msgr2=0x55e5c53bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f8788ac4640 1 --2- 172.21.15.78:0/728640691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b3440 0x55e5c53bd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f8788ac4640 1 -- 172.21.15.78:0/728640691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b8fa0 msgr2=0x55e5c53c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f8788ac4640 1 --2- 172.21.15.78:0/728640691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b8fa0 0x55e5c53c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f8788ac4640 1 -- 172.21.15.78:0/728640691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c50f56b0 con 0x55e5c53b3e30 2024-04-25T01:45:07.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f8789ac6640 1 --2- 172.21.15.78:0/728640691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b8fa0 0x55e5c53c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:45:07.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f8788ac4640 1 --2- 172.21.15.78:0/728640691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3e30 0x55e5c53bdd50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8774097990 tx=0x7f877409af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81e9928613bcfda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f876cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:07.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f876cff9640 1 -- 172.21.15.78:0/728640691 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8774020040 con 0x55e5c53b3e30 2024-04-25T01:45:07.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f876cff9640 1 -- 172.21.15.78:0/728640691 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f877409a980 con 0x55e5c53b3e30 2024-04-25T01:45:07.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f878e23a740 1 -- 172.21.15.78:0/728640691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f878004d070 con 0x55e5c53b3e30 2024-04-25T01:45:07.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f876cff9640 1 -- 172.21.15.78:0/728640691 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87740bb040 con 0x55e5c53b3e30 2024-04-25T01:45:07.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f878e23a740 10 client.15276 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:07.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f878e23a740 1 -- 172.21.15.78:0/728640691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5c53c7de0 con 0x55e5c53b3e30 2024-04-25T01:45:07.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f876cff9640 1 -- 172.21.15.78:0/728640691 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f878004d070 con 0x55e5c53b3e30 2024-04-25T01:45:07.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f876cff9640 1 -- 172.21.15.78:0/728640691 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8774084ac0 con 0x55e5c53b3e30 2024-04-25T01:45:07.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f878e23a740 10 client.15276 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:07.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f878e23a740 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T01:45:07.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f878e23a740 20 client.15276 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:07.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.319+0000 7f878e23a740 1 --2- 172.21.15.78:0/728640691 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55e5c53ca3c0 0x55e5c53cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.320+0000 7f878e23a740 4 client.15276 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:45:07.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.320+0000 7f878e23a740 1 -- 172.21.15.78:0/728640691 --> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5c5324440 con 0x55e5c53ca3c0 2024-04-25T01:45:07.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.320+0000 7f87892c5640 1 --2- 172.21.15.78:0/728640691 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55e5c53ca3c0 0x55e5c53cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.320+0000 7f876cff9640 10 client.15276 ms_handle_connect on v2:172.21.15.196:6838/460829167 2024-04-25T01:45:07.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.320+0000 7f87892c5640 1 --2- 172.21.15.78:0/728640691 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55e5c53ca3c0 0x55e5c53cb830 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c993e20af173787 server_cookie=2b78d06ee395bcfa in_seq=0 out_seq=0 2024-04-25T01:45:07.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f876cff9640 1 -- 172.21.15.78:0/728640691 <== mds.2 v2:172.21.15.196:6838/460829167 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5c5324440 con 0x55e5c53ca3c0 2024-04-25T01:45:07.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f876cff9640 10 client.15276 handle_command_reply: tid=0 2024-04-25T01:45:07.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f878e23a740 1 client.15276 shutdown 2024-04-25T01:45:07.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f878e23a740 1 -- 172.21.15.78:0/728640691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3e30 msgr2=0x55e5c53bdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f878e23a740 1 --2- 172.21.15.78:0/728640691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3e30 0x55e5c53bdd50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8774097990 tx=0x7f877409af50 comp rx=0 tx=0).stop 2024-04-25T01:45:07.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f878e23a740 1 -- 172.21.15.78:0/728640691 shutdown_connections 2024-04-25T01:45:07.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f878e23a740 1 --2- 172.21.15.78:0/728640691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b8fa0 0x55e5c53c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f878e23a740 1 --2- 172.21.15.78:0/728640691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3e30 0x55e5c53bdd50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f878e23a740 1 --2- 172.21.15.78:0/728640691 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55e5c53ca3c0 0x55e5c53cb830 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f878e23a740 1 --2- 172.21.15.78:0/728640691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b3440 0x55e5c53bd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f876cff9640 0 client.15276 ms_handle_reset on v2:172.21.15.196:6838/460829167 2024-04-25T01:45:07.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f878e23a740 1 -- 172.21.15.78:0/728640691 >> 172.21.15.78:0/728640691 conn(0x55e5c53aeab0 msgr2=0x55e5c53adae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f878e23a740 1 -- 172.21.15.78:0/728640691 shutdown_connections 2024-04-25T01:45:07.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f878e23a740 1 -- 172.21.15.78:0/728640691 wait complete. 2024-04-25T01:45:07.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.321+0000 7f878e23a740 20 client.15276 trim_cache size 0 max 16384 2024-04-25T01:45:07.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.354+0000 7f878e23a740 1 Processor -- start 2024-04-25T01:45:07.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.354+0000 7f878e23a740 1 -- start start 2024-04-25T01:45:07.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.354+0000 7f878e23a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b3440 0x55e5c53a9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.354+0000 7f878e23a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3e30 0x55e5c53c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.354+0000 7f878e23a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b8fa0 0x55e5c53bd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.354+0000 7f878e23a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e5c53c9b70 con 0x55e5c53b8fa0 2024-04-25T01:45:07.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.354+0000 7f878e23a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55e5c53ca240 con 0x55e5c53b3e30 2024-04-25T01:45:07.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.354+0000 7f878e23a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e5c53ca910 con 0x55e5c53b3440 2024-04-25T01:45:07.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f8789ac6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b8fa0 0x55e5c53bd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f8789ac6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b8fa0 0x55e5c53bd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41154/0 (socket says 172.21.15.78:41154) 2024-04-25T01:45:07.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f8789ac6640 1 -- 172.21.15.78:0/1189351006 learned_addr learned my addr 172.21.15.78:0/1189351006 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f8788ac4640 1 --2- 172.21.15.78:0/1189351006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3e30 0x55e5c53c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f87892c5640 1 --2- 172.21.15.78:0/1189351006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b3440 0x55e5c53a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f8789ac6640 1 -- 172.21.15.78:0/1189351006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b3440 msgr2=0x55e5c53a9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f8789ac6640 1 --2- 172.21.15.78:0/1189351006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b3440 0x55e5c53a9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f8789ac6640 1 -- 172.21.15.78:0/1189351006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3e30 msgr2=0x55e5c53c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f8789ac6640 1 --2- 172.21.15.78:0/1189351006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3e30 0x55e5c53c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f8789ac6640 1 -- 172.21.15.78:0/1189351006 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c53af780 con 0x55e5c53b8fa0 2024-04-25T01:45:07.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f87892c5640 1 --2- 172.21.15.78:0/1189351006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b3440 0x55e5c53a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:07.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f8788ac4640 1 --2- 172.21.15.78:0/1189351006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3e30 0x55e5c53c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:07.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f8789ac6640 1 --2- 172.21.15.78:0/1189351006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b8fa0 0x55e5c53bd810 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f877c00b760 tx=0x7f877c009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d64e6f3a198909 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f876effd640 1 -- 172.21.15.78:0/1189351006 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c007c20 con 0x55e5c53b8fa0 2024-04-25T01:45:07.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f876effd640 1 -- 172.21.15.78:0/1189351006 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c01b070 con 0x55e5c53b8fa0 2024-04-25T01:45:07.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.355+0000 7f876effd640 1 -- 172.21.15.78:0/1189351006 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c015710 con 0x55e5c53b8fa0 2024-04-25T01:45:07.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.356+0000 7f878e23a740 1 -- 172.21.15.78:0/1189351006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b8fa0 msgr2=0x55e5c53bd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.356+0000 7f878e23a740 1 --2- 172.21.15.78:0/1189351006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b8fa0 0x55e5c53bd810 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f877c00b760 tx=0x7f877c009670 comp rx=0 tx=0).stop 2024-04-25T01:45:07.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.356+0000 7f878e23a740 1 -- 172.21.15.78:0/1189351006 shutdown_connections 2024-04-25T01:45:07.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.356+0000 7f878e23a740 1 --2- 172.21.15.78:0/1189351006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b8fa0 0x55e5c53bd810 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.356+0000 7f878e23a740 1 --2- 172.21.15.78:0/1189351006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3e30 0x55e5c53c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.356+0000 7f878e23a740 1 --2- 172.21.15.78:0/1189351006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b3440 0x55e5c53a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.356+0000 7f878e23a740 1 -- 172.21.15.78:0/1189351006 >> 172.21.15.78:0/1189351006 conn(0x55e5c53aeab0 msgr2=0x55e5c53aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.356+0000 7f878e23a740 1 -- 172.21.15.78:0/1189351006 shutdown_connections 2024-04-25T01:45:07.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.356+0000 7f878e23a740 1 -- 172.21.15.78:0/1189351006 wait complete. 2024-04-25T01:45:07.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.356+0000 7f878e23a740 1 -- start start 2024-04-25T01:45:07.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.356+0000 7f878e23a740 1 Processor -- start 2024-04-25T01:45:07.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f878e23a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3440 0x55e5c53c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f878e23a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3e30 0x55e5c542c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f878e23a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b8fa0 0x55e5c542e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f878e23a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e5c53cb6b0 con 0x55e5c53b3e30 2024-04-25T01:45:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f878e23a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55e5c53cafe0 con 0x55e5c53b3440 2024-04-25T01:45:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f878e23a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e5c53cbd80 con 0x55e5c53b8fa0 2024-04-25T01:45:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f878e23a740 10 client.0 _ll_register_callbacks cb 0x7f877409b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f8788ac4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3e30 0x55e5c542c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f8788ac4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3e30 0x55e5c542c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41170/0 (socket says 172.21.15.78:41170) 2024-04-25T01:45:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f8788ac4640 1 -- 172.21.15.78:0/3249448537 learned_addr learned my addr 172.21.15.78:0/3249448537 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f8789ac6640 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b8fa0 0x55e5c542e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f87892c5640 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3440 0x55e5c53c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f8788ac4640 1 -- 172.21.15.78:0/3249448537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b8fa0 msgr2=0x55e5c542e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f8788ac4640 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b8fa0 0x55e5c542e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f8788ac4640 1 -- 172.21.15.78:0/3249448537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3440 msgr2=0x55e5c53c2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f8788ac4640 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3440 0x55e5c53c2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f8788ac4640 1 -- 172.21.15.78:0/3249448537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c53b8cf0 con 0x55e5c53b3e30 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f8789ac6640 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b8fa0 0x55e5c542e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f87892c5640 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3440 0x55e5c53c2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f8788ac4640 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3e30 0x55e5c542c350 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8774009610 tx=0x7f877409afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc0d2f1f1b2a453 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f876cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f876cff9640 1 -- 172.21.15.78:0/3249448537 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8774020040 con 0x55e5c53b3e30 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f876cff9640 1 -- 172.21.15.78:0/3249448537 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f8774097280 con 0x55e5c53b3e30 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.357+0000 7f878e23a740 1 -- 172.21.15.78:0/3249448537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f877c01b070 con 0x55e5c53b3e30 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.358+0000 7f876cff9640 1 -- 172.21.15.78:0/3249448537 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87740a41a0 con 0x55e5c53b3e30 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.358+0000 7f878e23a740 10 client.15282 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.358+0000 7f878e23a740 1 -- 172.21.15.78:0/3249448537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5c54301a0 con 0x55e5c53b3e30 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.358+0000 7f876cff9640 1 -- 172.21.15.78:0/3249448537 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c01b070 con 0x55e5c53b3e30 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.358+0000 7f876cff9640 1 -- 172.21.15.78:0/3249448537 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f877409a590 con 0x55e5c53b3e30 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.358+0000 7f878e23a740 10 client.15282 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.358+0000 7f878e23a740 10 client.15282 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T01:45:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.358+0000 7f878e23a740 20 client.15282 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.358+0000 7f878e23a740 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55e5c5434c70 0x55e5c5433980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.358+0000 7f878e23a740 4 client.15282 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:45:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.358+0000 7f878e23a740 1 -- 172.21.15.78:0/3249448537 --> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] -- command(tid 0: {"prefix": "status"}) -- 0x55e5c531e790 con 0x55e5c5434c70 2024-04-25T01:45:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.358+0000 7f87892c5640 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55e5c5434c70 0x55e5c5433980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f87892c5640 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55e5c5434c70 0x55e5c5433980 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c332a4add6ebbb5d server_cookie=665cecaf23fba8d4 in_seq=0 out_seq=0 2024-04-25T01:45:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f876cff9640 10 client.15282 ms_handle_connect on v2:172.21.15.196:6838/460829167 2024-04-25T01:45:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f876cff9640 1 -- 172.21.15.78:0/3249448537 <== mds.2 v2:172.21.15.196:6838/460829167 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e5c531e790 con 0x55e5c5434c70 2024-04-25T01:45:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f876cff9640 10 client.15282 handle_command_reply: tid=0 2024-04-25T01:45:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f878e23a740 1 client.15282 shutdown 2024-04-25T01:45:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f878e23a740 1 -- 172.21.15.78:0/3249448537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3e30 msgr2=0x55e5c542c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f878e23a740 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3e30 0x55e5c542c350 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8774009610 tx=0x7f877409afc0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f878e23a740 1 -- 172.21.15.78:0/3249448537 shutdown_connections 2024-04-25T01:45:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f878e23a740 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55e5c5434c70 0x55e5c5433980 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f878e23a740 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e5c53b8fa0 0x55e5c542e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f878e23a740 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5c53b3e30 0x55e5c542c350 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f878e23a740 1 --2- 172.21.15.78:0/3249448537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e5c53b3440 0x55e5c53c2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f876cff9640 0 client.15282 ms_handle_reset on v2:172.21.15.196:6838/460829167 2024-04-25T01:45:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f878e23a740 1 -- 172.21.15.78:0/3249448537 >> 172.21.15.78:0/3249448537 conn(0x55e5c53aeab0 msgr2=0x55e5c53a9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f878e23a740 1 -- 172.21.15.78:0/3249448537 shutdown_connections 2024-04-25T01:45:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f878e23a740 1 -- 172.21.15.78:0/3249448537 wait complete. 2024-04-25T01:45:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.359+0000 7f878e23a740 20 client.15282 trim_cache size 0 max 16384 2024-04-25T01:45:07.359 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:45:07.359 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "df36bea8-02a3-11ef-bc93-c7b262605968", 2024-04-25T01:45:07.359 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 2, 2024-04-25T01:45:07.359 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24448, 2024-04-25T01:45:07.359 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T01:45:07.360 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T01:45:07.360 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T01:45:07.360 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 85.362176429000002, 2024-04-25T01:45:07.360 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-04-25T01:45:07.360 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2024-04-25T01:45:07.360 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T01:45:07.360 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 123.98255077500001 2024-04-25T01:45:07.360 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:45:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.360+0000 7f878b529640 1 -- 172.21.15.78:0/1596261995 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f87700a2b90 msgr2=0x7f87700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.360+0000 7f878b529640 1 --2- 172.21.15.78:0/1596261995 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f87700a2b90 0x7f87700a5050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8780005ab0 tx=0x7f8780000bc0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.360+0000 7f878b529640 1 -- 172.21.15.78:0/1596261995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784101870 msgr2=0x7f8784198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.360+0000 7f878b529640 1 --2- 172.21.15.78:0/1596261995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784101870 0x7f8784198060 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f877400d950 tx=0x7f877400de10 comp rx=0 tx=0).stop 2024-04-25T01:45:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.360+0000 7f878b529640 1 -- 172.21.15.78:0/1596261995 shutdown_connections 2024-04-25T01:45:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.360+0000 7f878b529640 1 --2- 172.21.15.78:0/1596261995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8784102230 0x7f878419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.360+0000 7f878b529640 1 --2- 172.21.15.78:0/1596261995 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f87700a2b90 0x7f87700a5050 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.360+0000 7f878b529640 1 --2- 172.21.15.78:0/1596261995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8784101870 0x7f8784198060 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.360+0000 7f878b529640 1 --2- 172.21.15.78:0/1596261995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8784100680 0x7f8784197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.360+0000 7f878b529640 1 -- 172.21.15.78:0/1596261995 >> 172.21.15.78:0/1596261995 conn(0x7f87840fbbf0 msgr2=0x7f87840fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.360+0000 7f878b529640 1 -- 172.21.15.78:0/1596261995 shutdown_connections 2024-04-25T01:45:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.360+0000 7f878b529640 1 -- 172.21.15.78:0/1596261995 wait complete. 2024-04-25T01:45:07.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T01:45:07.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:07 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3248817840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:07.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:07 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2097597276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:07.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:07 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1190279703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.470+0000 7effcb7c1640 1 -- 172.21.15.78:0/3865529752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc4100680 msgr2=0x7effc4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.470+0000 7effcb7c1640 1 --2- 172.21.15.78:0/3865529752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc4100680 0x7effc4100aa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7effc0009900 tx=0x7effc002a410 comp rx=0 tx=0).stop 2024-04-25T01:45:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.470+0000 7effcb7c1640 1 -- 172.21.15.78:0/3865529752 shutdown_connections 2024-04-25T01:45:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.470+0000 7effcb7c1640 1 --2- 172.21.15.78:0/3865529752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc4102230 0x7effc4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.470+0000 7effcb7c1640 1 --2- 172.21.15.78:0/3865529752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc4101870 0x7effc4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.471+0000 7effcb7c1640 1 --2- 172.21.15.78:0/3865529752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc4100680 0x7effc4100aa0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.471+0000 7effcb7c1640 1 -- 172.21.15.78:0/3865529752 >> 172.21.15.78:0/3865529752 conn(0x7effc40fbbf0 msgr2=0x7effc40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.471+0000 7effcb7c1640 1 -- 172.21.15.78:0/3865529752 shutdown_connections 2024-04-25T01:45:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.471+0000 7effcb7c1640 1 -- 172.21.15.78:0/3865529752 wait complete. 2024-04-25T01:45:07.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.471+0000 7effcb7c1640 1 Processor -- start 2024-04-25T01:45:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.471+0000 7effcb7c1640 1 -- start start 2024-04-25T01:45:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.471+0000 7effcb7c1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc4100680 0x7effc41969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.471+0000 7effcb7c1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc4101870 0x7effc4196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.471+0000 7effcb7c1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc4102230 0x7effc419df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.471+0000 7effcb7c1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7effc410c300 con 0x7effc4102230 2024-04-25T01:45:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.471+0000 7effcb7c1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7effc410c180 con 0x7effc4101870 2024-04-25T01:45:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.471+0000 7effcb7c1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7effc410c480 con 0x7effc4100680 2024-04-25T01:45:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.471+0000 7effc9d5e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc4102230 0x7effc419df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.471+0000 7effc9d5e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc4102230 0x7effc419df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41186/0 (socket says 172.21.15.78:41186) 2024-04-25T01:45:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effc9d5e640 1 -- 172.21.15.78:0/3633597311 learned_addr learned my addr 172.21.15.78:0/3633597311 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effc8d5c640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc4101870 0x7effc4196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effc955d640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc4100680 0x7effc41969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effc9d5e640 1 -- 172.21.15.78:0/3633597311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc4100680 msgr2=0x7effc41969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effc9d5e640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc4100680 0x7effc41969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effc9d5e640 1 -- 172.21.15.78:0/3633597311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc4101870 msgr2=0x7effc4196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effc9d5e640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc4101870 0x7effc4196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effc9d5e640 1 -- 172.21.15.78:0/3633597311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc419e5f0 con 0x7effc4102230 2024-04-25T01:45:07.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effc955d640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc4100680 0x7effc41969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:07.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effc9d5e640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc4102230 0x7effc419df80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7effbc00eaa0 tx=0x7effbc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effba7fc640 1 -- 172.21.15.78:0/3633597311 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc00cdb0 con 0x7effc4102230 2024-04-25T01:45:07.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effcb7c1640 1 -- 172.21.15.78:0/3633597311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc419e8e0 con 0x7effc4102230 2024-04-25T01:45:07.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effba7fc640 1 -- 172.21.15.78:0/3633597311 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effbc005020 con 0x7effc4102230 2024-04-25T01:45:07.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effba7fc640 1 -- 172.21.15.78:0/3633597311 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc0116b0 con 0x7effc4102230 2024-04-25T01:45:07.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.472+0000 7effcb7c1640 1 -- 172.21.15.78:0/3633597311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc419ee30 con 0x7effc4102230 2024-04-25T01:45:07.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.475+0000 7effce5ab740 1 Processor -- start 2024-04-25T01:45:07.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.475+0000 7effba7fc640 1 -- 172.21.15.78:0/3633597311 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7effbc005560 con 0x7effc4102230 2024-04-25T01:45:07.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.475+0000 7effce5ab740 1 -- start start 2024-04-25T01:45:07.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.475+0000 7effce5ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68b650 0x557a2b68ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.475+0000 7effce5ab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68c040 0x557a2b690b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.475+0000 7effce5ab740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b6911b0 0x557a2b693640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.475+0000 7effce5ab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557a2b5ecf00 con 0x557a2b68c040 2024-04-25T01:45:07.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.475+0000 7effce5ab740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557a2b595750 con 0x557a2b6911b0 2024-04-25T01:45:07.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.475+0000 7effce5ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a2b5e8ec0 con 0x557a2b68b650 2024-04-25T01:45:07.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effc955d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68c040 0x557a2b690b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effc955d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68c040 0x557a2b690b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41192/0 (socket says 172.21.15.78:41192) 2024-04-25T01:45:07.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effc955d640 1 -- 172.21.15.78:0/3175271785 learned_addr learned my addr 172.21.15.78:0/3175271785 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effc8d5c640 1 --2- 172.21.15.78:0/3175271785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b6911b0 0x557a2b693640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effc8d5c640 1 --2- 172.21.15.78:0/3175271785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68b650 0x557a2b68ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effc955d640 1 -- 172.21.15.78:0/3175271785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68b650 msgr2=0x557a2b68ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effc955d640 1 --2- 172.21.15.78:0/3175271785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68b650 0x557a2b68ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effc955d640 1 -- 172.21.15.78:0/3175271785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b6911b0 msgr2=0x557a2b693640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effc955d640 1 --2- 172.21.15.78:0/3175271785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b6911b0 0x557a2b693640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effc955d640 1 -- 172.21.15.78:0/3175271785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a2b3012d0 con 0x557a2b68c040 2024-04-25T01:45:07.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effc8d5c640 1 --2- 172.21.15.78:0/3175271785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68b650 0x557a2b68ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:07.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effba7fc640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7effb00a2b90 0x7effb00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effc955d640 1 --2- 172.21.15.78:0/3175271785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68c040 0x557a2b690b40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7effc0031440 tx=0x7effc0032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab641ea7c6b70ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effc8d5c640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7effb00a2b90 0x7effb00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effba7fc640 1 -- 172.21.15.78:0/3633597311 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7effbc0cf5f0 con 0x7effc4102230 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effaf7fe640 1 -- 172.21.15.78:0/3175271785 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc003f070 con 0x557a2b68c040 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effaf7fe640 1 -- 172.21.15.78:0/3175271785 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc00318b0 con 0x557a2b68c040 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.476+0000 7effaf7fe640 1 -- 172.21.15.78:0/3175271785 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0035070 con 0x557a2b68c040 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.477+0000 7effc8d5c640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7effb00a2b90 0x7effb00a5050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x557a2b68c8a0 tx=0x7effb40096c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.477+0000 7effce5ab740 1 -- 172.21.15.78:0/3175271785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68c040 msgr2=0x557a2b690b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.477+0000 7effce5ab740 1 --2- 172.21.15.78:0/3175271785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68c040 0x557a2b690b40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7effc0031440 tx=0x7effc0032cb0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.477+0000 7effce5ab740 1 -- 172.21.15.78:0/3175271785 shutdown_connections 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.477+0000 7effce5ab740 1 --2- 172.21.15.78:0/3175271785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b6911b0 0x557a2b693640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.477+0000 7effce5ab740 1 --2- 172.21.15.78:0/3175271785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68c040 0x557a2b690b40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.477+0000 7effce5ab740 1 --2- 172.21.15.78:0/3175271785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68b650 0x557a2b68ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.477+0000 7effce5ab740 1 -- 172.21.15.78:0/3175271785 >> 172.21.15.78:0/3175271785 conn(0x557a2b684ab0 msgr2=0x557a2b684eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.477+0000 7effce5ab740 1 -- 172.21.15.78:0/3175271785 shutdown_connections 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.477+0000 7effce5ab740 1 -- 172.21.15.78:0/3175271785 wait complete. 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.478+0000 7effce5ab740 1 -- start start 2024-04-25T01:45:07.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.478+0000 7effce5ab740 1 Processor -- start 2024-04-25T01:45:07.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.478+0000 7effce5ab740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b68b650 0x557a2b695a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.478+0000 7effce5ab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68c040 0x557a2b695f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.478+0000 7effce5ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b6911b0 0x557a2b69cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.478+0000 7effce5ab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557a2b694100 con 0x557a2b68c040 2024-04-25T01:45:07.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.478+0000 7effce5ab740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557a2b5a8cf0 con 0x557a2b68b650 2024-04-25T01:45:07.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.478+0000 7effce5ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a2b694280 con 0x557a2b6911b0 2024-04-25T01:45:07.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.478+0000 7effce5ab740 10 client.0 _ll_register_callbacks cb 0x557a2b5a86e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:07.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.478+0000 7effc8d5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68c040 0x557a2b695f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.478+0000 7effc8d5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68c040 0x557a2b695f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41206/0 (socket says 172.21.15.78:41206) 2024-04-25T01:45:07.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.478+0000 7effc8d5c640 1 -- 172.21.15.78:0/3972983764 learned_addr learned my addr 172.21.15.78:0/3972983764 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effc955d640 1 --2- 172.21.15.78:0/3972983764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b68b650 0x557a2b695a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effc9d5e640 1 --2- 172.21.15.78:0/3972983764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b6911b0 0x557a2b69cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effc8d5c640 1 -- 172.21.15.78:0/3972983764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b6911b0 msgr2=0x557a2b69cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effc8d5c640 1 --2- 172.21.15.78:0/3972983764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b6911b0 0x557a2b69cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effc8d5c640 1 -- 172.21.15.78:0/3972983764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b68b650 msgr2=0x557a2b695a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effc8d5c640 1 --2- 172.21.15.78:0/3972983764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b68b650 0x557a2b695a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effc8d5c640 1 -- 172.21.15.78:0/3972983764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a2b3cb6b0 con 0x557a2b68c040 2024-04-25T01:45:07.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effc955d640 1 --2- 172.21.15.78:0/3972983764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b68b650 0x557a2b695a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:07.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effc8d5c640 1 --2- 172.21.15.78:0/3972983764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68c040 0x557a2b695f50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7effb401d9d0 tx=0x7effb4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66df53218419ddf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effacff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:07.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effacff9640 1 -- 172.21.15.78:0/3972983764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4028410 con 0x557a2b68c040 2024-04-25T01:45:07.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effacff9640 1 -- 172.21.15.78:0/3972983764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7effb4017030 con 0x557a2b68c040 2024-04-25T01:45:07.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effce5ab740 1 -- 172.21.15.78:0/3972983764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effc00318b0 con 0x557a2b68c040 2024-04-25T01:45:07.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effacff9640 1 -- 172.21.15.78:0/3972983764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effb4033050 con 0x557a2b68c040 2024-04-25T01:45:07.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effce5ab740 10 client.15294 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:07.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effce5ab740 1 -- 172.21.15.78:0/3972983764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557a2b69ffa0 con 0x557a2b68c040 2024-04-25T01:45:07.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effacff9640 1 -- 172.21.15.78:0/3972983764 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc00318b0 con 0x557a2b68c040 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.479+0000 7effacff9640 1 -- 172.21.15.78:0/3972983764 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7effb400f680 con 0x557a2b68c040 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.480+0000 7effce5ab740 10 client.15294 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.480+0000 7effce5ab740 10 client.15294 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.480+0000 7effce5ab740 20 client.15294 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.480+0000 7effce5ab740 1 --2- 172.21.15.78:0/3972983764 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x557a2b6a26d0 0x557a2b6a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.480+0000 7effce5ab740 4 client.15294 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.480+0000 7effce5ab740 1 -- 172.21.15.78:0/3972983764 --> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a2b5fa440 con 0x557a2b6a26d0 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.480+0000 7effc955d640 1 --2- 172.21.15.78:0/3972983764 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x557a2b6a26d0 0x557a2b6a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.480+0000 7effc955d640 1 --2- 172.21.15.78:0/3972983764 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x557a2b6a26d0 0x557a2b6a3b40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad6323f5da120627 server_cookie=d993126da6f50d77 in_seq=0 out_seq=0 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.480+0000 7effacff9640 10 client.15294 ms_handle_connect on v2:172.21.15.152:6836/4112159743 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.481+0000 7effacff9640 1 -- 172.21.15.78:0/3972983764 <== mds.1 v2:172.21.15.152:6836/4112159743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557a2b5fa440 con 0x557a2b6a26d0 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.481+0000 7effacff9640 10 client.15294 handle_command_reply: tid=0 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.481+0000 7effce5ab740 1 client.15294 shutdown 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.481+0000 7effce5ab740 1 -- 172.21.15.78:0/3972983764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68c040 msgr2=0x557a2b695f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.481+0000 7effce5ab740 1 --2- 172.21.15.78:0/3972983764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68c040 0x557a2b695f50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7effb401d9d0 tx=0x7effb4005f90 comp rx=0 tx=0).stop 2024-04-25T01:45:07.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.481+0000 7effce5ab740 1 -- 172.21.15.78:0/3972983764 shutdown_connections 2024-04-25T01:45:07.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.481+0000 7effce5ab740 1 --2- 172.21.15.78:0/3972983764 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x557a2b6a26d0 0x557a2b6a3b40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.481+0000 7effce5ab740 1 --2- 172.21.15.78:0/3972983764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b6911b0 0x557a2b69cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.481+0000 7effce5ab740 1 --2- 172.21.15.78:0/3972983764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68c040 0x557a2b695f50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.481+0000 7effce5ab740 1 --2- 172.21.15.78:0/3972983764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b68b650 0x557a2b695a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.481+0000 7effacff9640 0 client.15294 ms_handle_reset on v2:172.21.15.152:6836/4112159743 2024-04-25T01:45:07.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.481+0000 7effce5ab740 1 -- 172.21.15.78:0/3972983764 >> 172.21.15.78:0/3972983764 conn(0x557a2b684ab0 msgr2=0x557a2b683ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.481+0000 7effce5ab740 1 -- 172.21.15.78:0/3972983764 shutdown_connections 2024-04-25T01:45:07.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.481+0000 7effce5ab740 1 -- 172.21.15.78:0/3972983764 wait complete. 2024-04-25T01:45:07.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.482+0000 7effce5ab740 20 client.15294 trim_cache size 0 max 16384 2024-04-25T01:45:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.499+0000 7effce5ab740 1 Processor -- start 2024-04-25T01:45:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.499+0000 7effce5ab740 1 -- start start 2024-04-25T01:45:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effce5ab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68b650 0x557a2b684260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effce5ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68c040 0x557a2b69cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effce5ab740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b6911b0 0x557a2b67f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effce5ab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557a2b6a1e80 con 0x557a2b68b650 2024-04-25T01:45:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effce5ab740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557a2b6a2550 con 0x557a2b6911b0 2024-04-25T01:45:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effce5ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a2b6a2c20 con 0x557a2b68c040 2024-04-25T01:45:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effc955d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68b650 0x557a2b684260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effc955d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68b650 0x557a2b684260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41212/0 (socket says 172.21.15.78:41212) 2024-04-25T01:45:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effc955d640 1 -- 172.21.15.78:0/2159492361 learned_addr learned my addr 172.21.15.78:0/2159492361 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effc955d640 1 -- 172.21.15.78:0/2159492361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68c040 msgr2=0x557a2b69cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effc955d640 1 --2- 172.21.15.78:0/2159492361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68c040 0x557a2b69cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effc955d640 1 -- 172.21.15.78:0/2159492361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b6911b0 msgr2=0x557a2b67f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effc955d640 1 --2- 172.21.15.78:0/2159492361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b6911b0 0x557a2b67f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effc955d640 1 -- 172.21.15.78:0/2159492361 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a2b690f00 con 0x557a2b68b650 2024-04-25T01:45:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effc955d640 1 --2- 172.21.15.78:0/2159492361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68b650 0x557a2b684260 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7effc00096c0 tx=0x7effc0032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980f253bb471209 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effaeffd640 1 -- 172.21.15.78:0/2159492361 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0053070 con 0x557a2b68b650 2024-04-25T01:45:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effaeffd640 1 -- 172.21.15.78:0/2159492361 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc004e930 con 0x557a2b68b650 2024-04-25T01:45:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.500+0000 7effaeffd640 1 -- 172.21.15.78:0/2159492361 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0035040 con 0x557a2b68b650 2024-04-25T01:45:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.501+0000 7effce5ab740 1 -- 172.21.15.78:0/2159492361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68b650 msgr2=0x557a2b684260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.501+0000 7effce5ab740 1 --2- 172.21.15.78:0/2159492361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68b650 0x557a2b684260 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7effc00096c0 tx=0x7effc0032f60 comp rx=0 tx=0).stop 2024-04-25T01:45:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.501+0000 7effce5ab740 1 -- 172.21.15.78:0/2159492361 shutdown_connections 2024-04-25T01:45:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.501+0000 7effce5ab740 1 --2- 172.21.15.78:0/2159492361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b6911b0 0x557a2b67f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.501+0000 7effce5ab740 1 --2- 172.21.15.78:0/2159492361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68c040 0x557a2b69cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.501+0000 7effce5ab740 1 --2- 172.21.15.78:0/2159492361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b68b650 0x557a2b684260 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.501+0000 7effce5ab740 1 -- 172.21.15.78:0/2159492361 >> 172.21.15.78:0/2159492361 conn(0x557a2b684ab0 msgr2=0x557a2b684e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.501+0000 7effce5ab740 1 -- 172.21.15.78:0/2159492361 shutdown_connections 2024-04-25T01:45:07.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.501+0000 7effce5ab740 1 -- 172.21.15.78:0/2159492361 wait complete. 2024-04-25T01:45:07.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.501+0000 7effce5ab740 1 -- start start 2024-04-25T01:45:07.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.501+0000 7effce5ab740 1 Processor -- start 2024-04-25T01:45:07.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effce5ab740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68b650 0x557a2b69af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effce5ab740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b68c040 0x557a2b704660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effce5ab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b6911b0 0x557a2b706bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effce5ab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557a2b6a39c0 con 0x557a2b6911b0 2024-04-25T01:45:07.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effce5ab740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557a2b6a32f0 con 0x557a2b68c040 2024-04-25T01:45:07.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effce5ab740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557a2b6a4090 con 0x557a2b68b650 2024-04-25T01:45:07.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effce5ab740 10 client.0 _ll_register_callbacks cb 0x7effb401d890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:07.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effc9d5e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b6911b0 0x557a2b706bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effc9d5e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b6911b0 0x557a2b706bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41220/0 (socket says 172.21.15.78:41220) 2024-04-25T01:45:07.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effc9d5e640 1 -- 172.21.15.78:0/3506075220 learned_addr learned my addr 172.21.15.78:0/3506075220 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effc8d5c640 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b68c040 0x557a2b704660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effc955d640 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68b650 0x557a2b69af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effc9d5e640 1 -- 172.21.15.78:0/3506075220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68b650 msgr2=0x557a2b69af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effc9d5e640 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68b650 0x557a2b69af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effc9d5e640 1 -- 172.21.15.78:0/3506075220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b68c040 msgr2=0x557a2b704660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effc9d5e640 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b68c040 0x557a2b704660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effc9d5e640 1 -- 172.21.15.78:0/3506075220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a2b7070f0 con 0x557a2b6911b0 2024-04-25T01:45:07.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effc955d640 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68b650 0x557a2b69af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:07.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effc8d5c640 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b68c040 0x557a2b704660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:07.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effc9d5e640 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b6911b0 0x557a2b706bb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7effbc09baa0 tx=0x7effbc09c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65d834a713ad178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effacff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:07.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.502+0000 7effacff9640 1 -- 172.21.15.78:0/3506075220 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc01f040 con 0x557a2b6911b0 2024-04-25T01:45:07.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.503+0000 7effacff9640 1 -- 172.21.15.78:0/3506075220 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7effbc0051d0 con 0x557a2b6911b0 2024-04-25T01:45:07.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.503+0000 7effce5ab740 1 -- 172.21.15.78:0/3506075220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effc004e930 con 0x557a2b6911b0 2024-04-25T01:45:07.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.503+0000 7effacff9640 1 -- 172.21.15.78:0/3506075220 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effbc09af60 con 0x557a2b6911b0 2024-04-25T01:45:07.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.503+0000 7effce5ab740 10 client.15300 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:07.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.503+0000 7effce5ab740 1 -- 172.21.15.78:0/3506075220 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557a2b708460 con 0x557a2b6911b0 2024-04-25T01:45:07.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.503+0000 7effacff9640 1 -- 172.21.15.78:0/3506075220 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc004e930 con 0x557a2b6911b0 2024-04-25T01:45:07.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.503+0000 7effacff9640 1 -- 172.21.15.78:0/3506075220 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7effbc00f040 con 0x557a2b6911b0 2024-04-25T01:45:07.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.503+0000 7effce5ab740 10 client.15300 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:07.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.503+0000 7effce5ab740 10 client.15300 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T01:45:07.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.503+0000 7effce5ab740 20 client.15300 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.503+0000 7effce5ab740 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x557a2b70d030 0x557a2b70bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.503+0000 7effce5ab740 4 client.15300 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.503+0000 7effce5ab740 1 -- 172.21.15.78:0/3506075220 --> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] -- command(tid 0: {"prefix": "status"}) -- 0x557a2b5f4790 con 0x557a2b70d030 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.503+0000 7effc955d640 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x557a2b70d030 0x557a2b70bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.504+0000 7effc955d640 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x557a2b70d030 0x557a2b70bd40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf4d83daf6e3ffb9 server_cookie=b2ab6b5fdd81ee36 in_seq=0 out_seq=0 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.504+0000 7effacff9640 10 client.15300 ms_handle_connect on v2:172.21.15.152:6836/4112159743 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.504+0000 7effacff9640 1 -- 172.21.15.78:0/3506075220 <== mds.1 v2:172.21.15.152:6836/4112159743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557a2b5f4790 con 0x557a2b70d030 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.504+0000 7effacff9640 10 client.15300 handle_command_reply: tid=0 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.504+0000 7effce5ab740 1 client.15300 shutdown 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.504+0000 7effce5ab740 1 -- 172.21.15.78:0/3506075220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b6911b0 msgr2=0x557a2b706bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.504+0000 7effce5ab740 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b6911b0 0x557a2b706bb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7effbc09baa0 tx=0x7effbc09c350 comp rx=0 tx=0).stop 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.504+0000 7effce5ab740 1 -- 172.21.15.78:0/3506075220 shutdown_connections 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.504+0000 7effce5ab740 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x557a2b70d030 0x557a2b70bd40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.504+0000 7effce5ab740 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a2b6911b0 0x557a2b706bb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.504+0000 7effce5ab740 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557a2b68c040 0x557a2b704660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effce5ab740 1 --2- 172.21.15.78:0/3506075220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557a2b68b650 0x557a2b69af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effacff9640 0 client.15300 ms_handle_reset on v2:172.21.15.152:6836/4112159743 2024-04-25T01:45:07.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effce5ab740 1 -- 172.21.15.78:0/3506075220 >> 172.21.15.78:0/3506075220 conn(0x557a2b684ab0 msgr2=0x557a2b67f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effce5ab740 1 -- 172.21.15.78:0/3506075220 shutdown_connections 2024-04-25T01:45:07.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effce5ab740 1 -- 172.21.15.78:0/3506075220 wait complete. 2024-04-25T01:45:07.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effce5ab740 20 client.15300 trim_cache size 0 max 16384 2024-04-25T01:45:07.505 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:45:07.505 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "df36bea8-02a3-11ef-bc93-c7b262605968", 2024-04-25T01:45:07.505 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 1, 2024-04-25T01:45:07.505 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24455, 2024-04-25T01:45:07.505 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T01:45:07.506 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T01:45:07.506 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T01:45:07.506 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 86.514659072000001, 2024-04-25T01:45:07.506 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-04-25T01:45:07.506 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2024-04-25T01:45:07.506 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T01:45:07.506 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 126.88479985799999 2024-04-25T01:45:07.506 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:45:07.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effcb7c1640 1 -- 172.21.15.78:0/3633597311 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7effb00a2b90 msgr2=0x7effb00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effcb7c1640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7effb00a2b90 0x7effb00a5050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x557a2b68c8a0 tx=0x7effb40096c0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effcb7c1640 1 -- 172.21.15.78:0/3633597311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc4102230 msgr2=0x7effc419df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effcb7c1640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc4102230 0x7effc419df80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7effbc00eaa0 tx=0x7effbc00ef60 comp rx=0 tx=0).stop 2024-04-25T01:45:07.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effcb7c1640 1 -- 172.21.15.78:0/3633597311 shutdown_connections 2024-04-25T01:45:07.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effcb7c1640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7effb00a2b90 0x7effb00a5050 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effcb7c1640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7effc4102230 0x7effc419df80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effcb7c1640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effc4101870 0x7effc4196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effcb7c1640 1 --2- 172.21.15.78:0/3633597311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc4100680 0x7effc41969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effcb7c1640 1 -- 172.21.15.78:0/3633597311 >> 172.21.15.78:0/3633597311 conn(0x7effc40fbbf0 msgr2=0x7effc4104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effcb7c1640 1 -- 172.21.15.78:0/3633597311 shutdown_connections 2024-04-25T01:45:07.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.505+0000 7effcb7c1640 1 -- 172.21.15.78:0/3633597311 wait complete. 2024-04-25T01:45:07.509 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T01:45:07.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.616+0000 7fa88e763640 1 -- 172.21.15.78:0/2129568147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888102250 msgr2=0x7fa888108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.616+0000 7fa88e763640 1 --2- 172.21.15.78:0/2129568147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888102250 0x7fa888108ae0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa880009ca0 tx=0x7fa88002a8c0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.616+0000 7fa88e763640 1 -- 172.21.15.78:0/2129568147 shutdown_connections 2024-04-25T01:45:07.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.616+0000 7fa88e763640 1 --2- 172.21.15.78:0/2129568147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888102250 0x7fa888108ae0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.616+0000 7fa88e763640 1 --2- 172.21.15.78:0/2129568147 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa888101890 0x7fa888101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.616+0000 7fa88e763640 1 --2- 172.21.15.78:0/2129568147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8881006a0 0x7fa888100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.616+0000 7fa88e763640 1 -- 172.21.15.78:0/2129568147 >> 172.21.15.78:0/2129568147 conn(0x7fa8880fbc10 msgr2=0x7fa8880fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.616+0000 7fa88e763640 1 -- 172.21.15.78:0/2129568147 shutdown_connections 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.616+0000 7fa88e763640 1 -- 172.21.15.78:0/2129568147 wait complete. 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa88e763640 1 Processor -- start 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa88e763640 1 -- start start 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa88e763640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8881006a0 0x7fa88806a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa88e763640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888101890 0x7fa88819c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa88e763640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa888102250 0x7fa88819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa88e763640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa88810c320 con 0x7fa888101890 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa88e763640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa88810c1a0 con 0x7fa888102250 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa88e763640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa88810c4a0 con 0x7fa8881006a0 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa8877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888101890 0x7fa88819c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa8877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888101890 0x7fa88819c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41240/0 (socket says 172.21.15.78:41240) 2024-04-25T01:45:07.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa8877fe640 1 -- 172.21.15.78:0/2061241930 learned_addr learned my addr 172.21.15.78:0/2061241930 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa887fff640 1 --2- 172.21.15.78:0/2061241930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8881006a0 0x7fa88806a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa88cd00640 1 --2- 172.21.15.78:0/2061241930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa888102250 0x7fa88819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa8877fe640 1 -- 172.21.15.78:0/2061241930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8881006a0 msgr2=0x7fa88806a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa8877fe640 1 --2- 172.21.15.78:0/2061241930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8881006a0 0x7fa88806a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa8877fe640 1 -- 172.21.15.78:0/2061241930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa888102250 msgr2=0x7fa88819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa8877fe640 1 --2- 172.21.15.78:0/2061241930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa888102250 0x7fa88819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa8877fe640 1 -- 172.21.15.78:0/2061241930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88819f6b0 con 0x7fa888101890 2024-04-25T01:45:07.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.617+0000 7fa8877fe640 1 --2- 172.21.15.78:0/2061241930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888101890 0x7fa88819c780 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00d770 tx=0x7fa87c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.618+0000 7fa8857fa640 1 -- 172.21.15.78:0/2061241930 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c017070 con 0x7fa888101890 2024-04-25T01:45:07.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.618+0000 7fa8857fa640 1 -- 172.21.15.78:0/2061241930 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa87c005500 con 0x7fa888101890 2024-04-25T01:45:07.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.618+0000 7fa8857fa640 1 -- 172.21.15.78:0/2061241930 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c004e00 con 0x7fa888101890 2024-04-25T01:45:07.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.618+0000 7fa88e763640 1 -- 172.21.15.78:0/2061241930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88819f9a0 con 0x7fa888101890 2024-04-25T01:45:07.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.618+0000 7fa88e763640 1 -- 172.21.15.78:0/2061241930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88819fea0 con 0x7fa888101890 2024-04-25T01:45:07.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.620+0000 7fa8857fa640 1 -- 172.21.15.78:0/2061241930 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa87c004fa0 con 0x7fa888101890 2024-04-25T01:45:07.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.620+0000 7fa891473740 1 Processor -- start 2024-04-25T01:45:07.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.620+0000 7fa891473740 1 -- start start 2024-04-25T01:45:07.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.620+0000 7fa891473740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 0x5615f36ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.620+0000 7fa891473740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36cee30 0x5615f36d3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.620+0000 7fa891473740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36d3fa0 0x5615f36d6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.620+0000 7fa891473740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615f3631f00 con 0x5615f36d3fa0 2024-04-25T01:45:07.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.620+0000 7fa891473740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5615f35da750 con 0x5615f36cee30 2024-04-25T01:45:07.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.620+0000 7fa891473740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615f362dec0 con 0x5615f36ce440 2024-04-25T01:45:07.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.620+0000 7fa8877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36d3fa0 0x5615f36d6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.620+0000 7fa8877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36d3fa0 0x5615f36d6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41252/0 (socket says 172.21.15.78:41252) 2024-04-25T01:45:07.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.620+0000 7fa8877fe640 1 -- 172.21.15.78:0/534486139 learned_addr learned my addr 172.21.15.78:0/534486139 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa8857fa640 1 --2- 172.21.15.78:0/2061241930 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa8700a2b90 0x7fa8700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa887fff640 1 --2- 172.21.15.78:0/534486139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36cee30 0x5615f36d3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa88cd00640 1 --2- 172.21.15.78:0/534486139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 0x5615f36ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa88cd00640 1 --2- 172.21.15.78:0/2061241930 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa8700a2b90 0x7fa8700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa8857fa640 1 -- 172.21.15.78:0/2061241930 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fa87c0cee00 con 0x7fa888101890 2024-04-25T01:45:07.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa8877fe640 1 -- 172.21.15.78:0/534486139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 msgr2=0x5615f36ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa8877fe640 1 --2- 172.21.15.78:0/534486139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 0x5615f36ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa8877fe640 1 -- 172.21.15.78:0/534486139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36cee30 msgr2=0x5615f36d3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa8877fe640 1 --2- 172.21.15.78:0/534486139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36cee30 0x5615f36d3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa8877fe640 1 -- 172.21.15.78:0/534486139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f33462d0 con 0x5615f36d3fa0 2024-04-25T01:45:07.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa887fff640 1 --2- 172.21.15.78:0/534486139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36cee30 0x5615f36d3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:45:07.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa8877fe640 1 --2- 172.21.15.78:0/534486139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36d3fa0 0x5615f36d6430 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa87c097cf0 tx=0x7fa87c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ad7075e4b0f720 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa88cd00640 1 --2- 172.21.15.78:0/2061241930 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa8700a2b90 0x7fa8700a5050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa880031c10 tx=0x7fa8800095c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa86e7fc640 1 -- 172.21.15.78:0/534486139 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c020030 con 0x5615f36d3fa0 2024-04-25T01:45:07.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa86e7fc640 1 -- 172.21.15.78:0/534486139 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa87c09ba20 con 0x5615f36d3fa0 2024-04-25T01:45:07.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.621+0000 7fa86e7fc640 1 -- 172.21.15.78:0/534486139 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c099620 con 0x5615f36d3fa0 2024-04-25T01:45:07.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.622+0000 7fa891473740 1 -- 172.21.15.78:0/534486139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36d3fa0 msgr2=0x5615f36d6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.622+0000 7fa891473740 1 --2- 172.21.15.78:0/534486139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36d3fa0 0x5615f36d6430 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa87c097cf0 tx=0x7fa87c009c30 comp rx=0 tx=0).stop 2024-04-25T01:45:07.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.622+0000 7fa891473740 1 -- 172.21.15.78:0/534486139 shutdown_connections 2024-04-25T01:45:07.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.622+0000 7fa891473740 1 --2- 172.21.15.78:0/534486139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36d3fa0 0x5615f36d6430 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.622+0000 7fa891473740 1 --2- 172.21.15.78:0/534486139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36cee30 0x5615f36d3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.622+0000 7fa891473740 1 --2- 172.21.15.78:0/534486139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 0x5615f36ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.622+0000 7fa891473740 1 -- 172.21.15.78:0/534486139 >> 172.21.15.78:0/534486139 conn(0x5615f36c9ab0 msgr2=0x5615f36c9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.622+0000 7fa891473740 1 -- 172.21.15.78:0/534486139 shutdown_connections 2024-04-25T01:45:07.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.622+0000 7fa891473740 1 -- 172.21.15.78:0/534486139 wait complete. 2024-04-25T01:45:07.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.622+0000 7fa891473740 1 -- start start 2024-04-25T01:45:07.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.622+0000 7fa891473740 1 Processor -- start 2024-04-25T01:45:07.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa891473740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 0x5615f36d8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa891473740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f36d8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa891473740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36d3fa0 0x5615f36dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa891473740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615f36d6ef0 con 0x5615f36cee30 2024-04-25T01:45:07.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa891473740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5615f35edcf0 con 0x5615f36d3fa0 2024-04-25T01:45:07.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa891473740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615f36d7070 con 0x5615f36ce440 2024-04-25T01:45:07.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa891473740 10 client.0 _ll_register_callbacks cb 0x5615f35ed6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:07.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa8877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f36d8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa8877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f36d8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41256/0 (socket says 172.21.15.78:41256) 2024-04-25T01:45:07.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa8877fe640 1 -- 172.21.15.78:0/1576773518 learned_addr learned my addr 172.21.15.78:0/1576773518 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa88cd00640 1 --2- 172.21.15.78:0/1576773518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36d3fa0 0x5615f36dfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa887fff640 1 --2- 172.21.15.78:0/1576773518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 0x5615f36d8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa8877fe640 1 -- 172.21.15.78:0/1576773518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 msgr2=0x5615f36d8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa8877fe640 1 --2- 172.21.15.78:0/1576773518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 0x5615f36d8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa8877fe640 1 -- 172.21.15.78:0/1576773518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36d3fa0 msgr2=0x5615f36dfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa8877fe640 1 --2- 172.21.15.78:0/1576773518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36d3fa0 0x5615f36dfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa8877fe640 1 -- 172.21.15.78:0/1576773518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f34106b0 con 0x5615f36cee30 2024-04-25T01:45:07.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa887fff640 1 --2- 172.21.15.78:0/1576773518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 0x5615f36d8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:07.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa8877fe640 1 --2- 172.21.15.78:0/1576773518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f36d8d40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa87c09b2b0 tx=0x7fa87c0047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6201bd2539dfaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa8677fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:07.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa8677fe640 1 -- 172.21.15.78:0/1576773518 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c020070 con 0x5615f36cee30 2024-04-25T01:45:07.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa8677fe640 1 -- 172.21.15.78:0/1576773518 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fa87c009cc0 con 0x5615f36cee30 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa891473740 1 -- 172.21.15.78:0/1576773518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa87c09ba20 con 0x5615f36cee30 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa8677fe640 1 -- 172.21.15.78:0/1576773518 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa87c097990 con 0x5615f36cee30 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa891473740 10 client.15312 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.623+0000 7fa891473740 1 -- 172.21.15.78:0/1576773518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615f36e2d60 con 0x5615f36cee30 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.624+0000 7fa8677fe640 1 -- 172.21.15.78:0/1576773518 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa87c09ba20 con 0x5615f36cee30 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.624+0000 7fa8677fe640 1 -- 172.21.15.78:0/1576773518 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa87c09a770 con 0x5615f36cee30 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.624+0000 7fa891473740 10 client.15312 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.624+0000 7fa891473740 10 client.15312 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.624+0000 7fa891473740 20 client.15312 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.624+0000 7fa891473740 1 --2- 172.21.15.78:0/1576773518 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5615f36e4fa0 0x5615f36e6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.624+0000 7fa891473740 4 client.15312 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.624+0000 7fa891473740 1 -- 172.21.15.78:0/1576773518 --> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615f363f440 con 0x5615f36e4fa0 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.624+0000 7fa887fff640 1 --2- 172.21.15.78:0/1576773518 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5615f36e4fa0 0x5615f36e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.624+0000 7fa887fff640 1 --2- 172.21.15.78:0/1576773518 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5615f36e4fa0 0x5615f36e6410 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=271b95c76b9f0d95 server_cookie=7ffa93be380afc16 in_seq=0 out_seq=0 2024-04-25T01:45:07.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.625+0000 7fa8677fe640 10 client.15312 ms_handle_connect on v2:172.21.15.152:6838/1969344843 2024-04-25T01:45:07.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.625+0000 7fa8677fe640 1 -- 172.21.15.78:0/1576773518 <== mds.0 v2:172.21.15.152:6838/1969344843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615f363f440 con 0x5615f36e4fa0 2024-04-25T01:45:07.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.625+0000 7fa8677fe640 10 client.15312 handle_command_reply: tid=0 2024-04-25T01:45:07.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.625+0000 7fa891473740 1 client.15312 shutdown 2024-04-25T01:45:07.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.625+0000 7fa891473740 1 -- 172.21.15.78:0/1576773518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 msgr2=0x5615f36d8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.625+0000 7fa891473740 1 --2- 172.21.15.78:0/1576773518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f36d8d40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa87c09b2b0 tx=0x7fa87c0047f0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.625+0000 7fa891473740 1 -- 172.21.15.78:0/1576773518 shutdown_connections 2024-04-25T01:45:07.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.625+0000 7fa891473740 1 --2- 172.21.15.78:0/1576773518 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5615f36e4fa0 0x5615f36e6410 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.625+0000 7fa891473740 1 --2- 172.21.15.78:0/1576773518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36d3fa0 0x5615f36dfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.625+0000 7fa891473740 1 --2- 172.21.15.78:0/1576773518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f36d8d40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.625+0000 7fa8677fe640 0 client.15312 ms_handle_reset on v2:172.21.15.152:6838/1969344843 2024-04-25T01:45:07.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.625+0000 7fa891473740 1 --2- 172.21.15.78:0/1576773518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 0x5615f36d8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.625+0000 7fa891473740 1 -- 172.21.15.78:0/1576773518 >> 172.21.15.78:0/1576773518 conn(0x5615f36c9ab0 msgr2=0x5615f36c8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.625+0000 7fa891473740 1 -- 172.21.15.78:0/1576773518 shutdown_connections 2024-04-25T01:45:07.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.626+0000 7fa891473740 1 -- 172.21.15.78:0/1576773518 wait complete. 2024-04-25T01:45:07.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.626+0000 7fa891473740 20 client.15312 trim_cache size 0 max 16384 2024-04-25T01:45:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.643+0000 7fa891473740 1 Processor -- start 2024-04-25T01:45:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.643+0000 7fa891473740 1 -- start start 2024-04-25T01:45:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.643+0000 7fa891473740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 0x5615f36d7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.643+0000 7fa891473740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f36dfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.643+0000 7fa891473740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36d3fa0 0x5615f36e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.643+0000 7fa891473740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615f36e4750 con 0x5615f36cee30 2024-04-25T01:45:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.643+0000 7fa891473740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5615f36e4e20 con 0x5615f36d3fa0 2024-04-25T01:45:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.643+0000 7fa891473740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615f36e54f0 con 0x5615f36ce440 2024-04-25T01:45:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.643+0000 7fa8877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f36dfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.644+0000 7fa8877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f36dfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41260/0 (socket says 172.21.15.78:41260) 2024-04-25T01:45:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.644+0000 7fa8877fe640 1 -- 172.21.15.78:0/1686477935 learned_addr learned my addr 172.21.15.78:0/1686477935 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.644+0000 7fa887fff640 1 --2- 172.21.15.78:0/1686477935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 0x5615f36d7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.644+0000 7fa8877fe640 1 -- 172.21.15.78:0/1686477935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 msgr2=0x5615f36d7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.644+0000 7fa8877fe640 1 --2- 172.21.15.78:0/1686477935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 0x5615f36d7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.644+0000 7fa8877fe640 1 -- 172.21.15.78:0/1686477935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36d3fa0 msgr2=0x5615f36e40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.644+0000 7fa8877fe640 1 --2- 172.21.15.78:0/1686477935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36d3fa0 0x5615f36e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.644+0000 7fa8877fe640 1 -- 172.21.15.78:0/1686477935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f36e6740 con 0x5615f36cee30 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.644+0000 7fa887fff640 1 --2- 172.21.15.78:0/1686477935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 0x5615f36d7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.644+0000 7fa8877fe640 1 --2- 172.21.15.78:0/1686477935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f36dfd40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0995d0 tx=0x7fa87c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bafde12d77a02ca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.644+0000 7fa86dffb640 1 -- 172.21.15.78:0/1686477935 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c020070 con 0x5615f36cee30 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.644+0000 7fa86dffb640 1 -- 172.21.15.78:0/1686477935 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa87c09ba20 con 0x5615f36cee30 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.644+0000 7fa86dffb640 1 -- 172.21.15.78:0/1686477935 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0b4020 con 0x5615f36cee30 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 -- 172.21.15.78:0/1686477935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 msgr2=0x5615f36dfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 --2- 172.21.15.78:0/1686477935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f36dfd40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0995d0 tx=0x7fa87c009c30 comp rx=0 tx=0).stop 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 -- 172.21.15.78:0/1686477935 shutdown_connections 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 --2- 172.21.15.78:0/1686477935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36d3fa0 0x5615f36e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 --2- 172.21.15.78:0/1686477935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f36dfd40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 --2- 172.21.15.78:0/1686477935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36ce440 0x5615f36d7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 -- 172.21.15.78:0/1686477935 >> 172.21.15.78:0/1686477935 conn(0x5615f36c9ab0 msgr2=0x5615f36c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 -- 172.21.15.78:0/1686477935 shutdown_connections 2024-04-25T01:45:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 -- 172.21.15.78:0/1686477935 wait complete. 2024-04-25T01:45:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 -- start start 2024-04-25T01:45:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 Processor -- start 2024-04-25T01:45:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36ce440 0x5615f36ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f3746f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36d3fa0 0x5615f3749480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615f36e6290 con 0x5615f36cee30 2024-04-25T01:45:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5615f36e5bc0 con 0x5615f36ce440 2024-04-25T01:45:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615f36e6960 con 0x5615f36d3fa0 2024-04-25T01:45:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa891473740 10 client.0 _ll_register_callbacks cb 0x7fa87c09ad00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa8877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f3746f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa8877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f3746f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41264/0 (socket says 172.21.15.78:41264) 2024-04-25T01:45:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.645+0000 7fa8877fe640 1 -- 172.21.15.78:0/2397471090 learned_addr learned my addr 172.21.15.78:0/2397471090 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa887fff640 1 --2- 172.21.15.78:0/2397471090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36ce440 0x5615f36ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa88cd00640 1 --2- 172.21.15.78:0/2397471090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36d3fa0 0x5615f3749480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa8877fe640 1 -- 172.21.15.78:0/2397471090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36d3fa0 msgr2=0x5615f3749480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa8877fe640 1 --2- 172.21.15.78:0/2397471090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36d3fa0 0x5615f3749480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa8877fe640 1 -- 172.21.15.78:0/2397471090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36ce440 msgr2=0x5615f36ddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa8877fe640 1 --2- 172.21.15.78:0/2397471090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36ce440 0x5615f36ddc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa8877fe640 1 -- 172.21.15.78:0/2397471090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f3749a40 con 0x5615f36cee30 2024-04-25T01:45:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa887fff640 1 --2- 172.21.15.78:0/2397471090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36ce440 0x5615f36ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa8877fe640 1 --2- 172.21.15.78:0/2397471090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f3746f30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0bb3a0 tx=0x7fa87c095d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c608d8b2c57556b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa8677fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa8677fe640 1 -- 172.21.15.78:0/2397471090 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c020040 con 0x5615f36cee30 2024-04-25T01:45:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa8677fe640 1 -- 172.21.15.78:0/2397471090 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fa87c0972f0 con 0x5615f36cee30 2024-04-25T01:45:07.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa891473740 1 -- 172.21.15.78:0/2397471090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa87c09ba20 con 0x5615f36cee30 2024-04-25T01:45:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa8677fe640 1 -- 172.21.15.78:0/2397471090 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa87c09f6d0 con 0x5615f36cee30 2024-04-25T01:45:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa891473740 10 client.15318 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa891473740 1 -- 172.21.15.78:0/2397471090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615f3749a40 con 0x5615f36cee30 2024-04-25T01:45:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa8677fe640 1 -- 172.21.15.78:0/2397471090 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa87c09ba20 con 0x5615f36cee30 2024-04-25T01:45:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa8677fe640 1 -- 172.21.15.78:0/2397471090 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa87c0a0860 con 0x5615f36cee30 2024-04-25T01:45:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa891473740 10 client.15318 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa891473740 10 client.15318 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T01:45:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa891473740 20 client.15318 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa891473740 1 --2- 172.21.15.78:0/2397471090 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5615f374f270 0x5615f374df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa891473740 4 client.15318 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:45:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.646+0000 7fa891473740 1 -- 172.21.15.78:0/2397471090 --> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] -- command(tid 0: {"prefix": "status"}) -- 0x5615f3639790 con 0x5615f374f270 2024-04-25T01:45:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.647+0000 7fa887fff640 1 --2- 172.21.15.78:0/2397471090 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5615f374f270 0x5615f374df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.647+0000 7fa887fff640 1 --2- 172.21.15.78:0/2397471090 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5615f374f270 0x5615f374df80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ee4fb87fae8a4d6 server_cookie=b6577e75fb03b856 in_seq=0 out_seq=0 2024-04-25T01:45:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.647+0000 7fa8677fe640 10 client.15318 ms_handle_connect on v2:172.21.15.152:6838/1969344843 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.647+0000 7fa8677fe640 1 -- 172.21.15.78:0/2397471090 <== mds.0 v2:172.21.15.152:6838/1969344843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5615f3639790 con 0x5615f374f270 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.647+0000 7fa8677fe640 10 client.15318 handle_command_reply: tid=0 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.647+0000 7fa891473740 1 client.15318 shutdown 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa891473740 1 -- 172.21.15.78:0/2397471090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 msgr2=0x5615f3746f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa891473740 1 --2- 172.21.15.78:0/2397471090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f3746f30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0bb3a0 tx=0x7fa87c095d60 comp rx=0 tx=0).stop 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa891473740 1 -- 172.21.15.78:0/2397471090 shutdown_connections 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa891473740 1 --2- 172.21.15.78:0/2397471090 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x5615f374f270 0x5615f374df80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa891473740 1 --2- 172.21.15.78:0/2397471090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615f36d3fa0 0x5615f3749480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa8677fe640 0 client.15318 ms_handle_reset on v2:172.21.15.152:6838/1969344843 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa891473740 1 --2- 172.21.15.78:0/2397471090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615f36cee30 0x5615f3746f30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa891473740 1 --2- 172.21.15.78:0/2397471090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5615f36ce440 0x5615f36ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa891473740 1 -- 172.21.15.78:0/2397471090 >> 172.21.15.78:0/2397471090 conn(0x5615f36c9ab0 msgr2=0x5615f36c9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa891473740 1 -- 172.21.15.78:0/2397471090 shutdown_connections 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa891473740 1 -- 172.21.15.78:0/2397471090 wait complete. 2024-04-25T01:45:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa891473740 20 client.15318 trim_cache size 0 max 16384 2024-04-25T01:45:07.648 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:45:07.648 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "df36bea8-02a3-11ef-bc93-c7b262605968", 2024-04-25T01:45:07.648 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-04-25T01:45:07.648 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24491, 2024-04-25T01:45:07.648 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T01:45:07.648 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T01:45:07.648 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T01:45:07.648 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 109.62105819200001, 2024-04-25T01:45:07.648 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-04-25T01:45:07.649 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 92, 2024-04-25T01:45:07.649 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T01:45:07.649 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 115.715946238 2024-04-25T01:45:07.649 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:45:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa88e763640 1 -- 172.21.15.78:0/2061241930 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa8700a2b90 msgr2=0x7fa8700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa88e763640 1 --2- 172.21.15.78:0/2061241930 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa8700a2b90 0x7fa8700a5050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa880031c10 tx=0x7fa8800095c0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa88e763640 1 -- 172.21.15.78:0/2061241930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888101890 msgr2=0x7fa88819c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa88e763640 1 --2- 172.21.15.78:0/2061241930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888101890 0x7fa88819c780 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00d770 tx=0x7fa87c00dc30 comp rx=0 tx=0).stop 2024-04-25T01:45:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa88e763640 1 -- 172.21.15.78:0/2061241930 shutdown_connections 2024-04-25T01:45:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa88e763640 1 --2- 172.21.15.78:0/2061241930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa888102250 0x7fa88819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa88e763640 1 --2- 172.21.15.78:0/2061241930 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fa8700a2b90 0x7fa8700a5050 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa88e763640 1 --2- 172.21.15.78:0/2061241930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888101890 0x7fa88819c780 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa88e763640 1 --2- 172.21.15.78:0/2061241930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8881006a0 0x7fa88806a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa88e763640 1 -- 172.21.15.78:0/2061241930 >> 172.21.15.78:0/2061241930 conn(0x7fa8880fbc10 msgr2=0x7fa888104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa88e763640 1 -- 172.21.15.78:0/2061241930 shutdown_connections 2024-04-25T01:45:07.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:07.648+0000 7fa88e763640 1 -- 172.21.15.78:0/2061241930 wait complete. 2024-04-25T01:45:07.652 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T01:45:07.652 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T01:45:07.662 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T01:45:07.662 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T01:45:07.900 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:45:08.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:08 smithi078 ceph-mon[29469]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:45:08.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:08 smithi152 ceph-mon[32465]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:45:08.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:08 smithi196 ceph-mon[32978]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:45:09.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.699+0000 7fab916d8700 1 -- 172.21.15.78:0/3834738108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c101190 msgr2=0x7fab8c1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:09.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.699+0000 7fab916d8700 1 --2- 172.21.15.78:0/3834738108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c101190 0x7fab8c1015b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fab800099c0 tx=0x7fab80009cd0 comp rx=0 tx=0).stop 2024-04-25T01:45:09.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.699+0000 7fab916d8700 1 -- 172.21.15.78:0/3834738108 shutdown_connections 2024-04-25T01:45:09.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.699+0000 7fab916d8700 1 --2- 172.21.15.78:0/3834738108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c102d30 0x7fab8c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:09.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.699+0000 7fab916d8700 1 --2- 172.21.15.78:0/3834738108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c101190 0x7fab8c1015b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:09.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.699+0000 7fab916d8700 1 --2- 172.21.15.78:0/3834738108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102390 0x7fab8c1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:09.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.699+0000 7fab916d8700 1 -- 172.21.15.78:0/3834738108 >> 172.21.15.78:0/3834738108 conn(0x7fab8c0fc750 msgr2=0x7fab8c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:09.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.699+0000 7fab916d8700 1 -- 172.21.15.78:0/3834738108 shutdown_connections 2024-04-25T01:45:09.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.700+0000 7fab916d8700 1 -- 172.21.15.78:0/3834738108 wait complete. 2024-04-25T01:45:09.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.700+0000 7fab916d8700 1 Processor -- start 2024-04-25T01:45:09.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.700+0000 7fab916d8700 1 -- start start 2024-04-25T01:45:09.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.700+0000 7fab916d8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c101190 0x7fab8c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:09.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.700+0000 7fab916d8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102390 0x7fab8c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:09.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.700+0000 7fab916d8700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c102d30 0x7fab8c19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:09.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.700+0000 7fab916d8700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c10cdf0 con 0x7fab8c102d30 2024-04-25T01:45:09.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.700+0000 7fab916d8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c10cc70 con 0x7fab8c101190 2024-04-25T01:45:09.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.700+0000 7fab916d8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c10cf70 con 0x7fab8c102390 2024-04-25T01:45:09.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab8b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c102d30 0x7fab8c19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:09.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab8b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c102d30 0x7fab8c19fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42664/0 (socket says 172.21.15.78:42664) 2024-04-25T01:45:09.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab8b7fe700 1 -- 172.21.15.78:0/1206130460 learned_addr learned my addr 172.21.15.78:0/1206130460 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:09.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab8a7fc700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102390 0x7fab8c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab8affd700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c101190 0x7fab8c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab8b7fe700 1 -- 172.21.15.78:0/1206130460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102390 msgr2=0x7fab8c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab8b7fe700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102390 0x7fab8c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab8b7fe700 1 -- 172.21.15.78:0/1206130460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c101190 msgr2=0x7fab8c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab8b7fe700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c101190 0x7fab8c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab8b7fe700 1 -- 172.21.15.78:0/1206130460 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c1a01a0 con 0x7fab8c102d30 2024-04-25T01:45:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab8affd700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c101190 0x7fab8c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab8b7fe700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c102d30 0x7fab8c19fb00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fab8400b9a0 tx=0x7fab8400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab73fff700 1 -- 172.21.15.78:0/1206130460 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84019070 con 0x7fab8c102d30 2024-04-25T01:45:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab73fff700 1 -- 172.21.15.78:0/1206130460 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8400ce10 con 0x7fab8c102d30 2024-04-25T01:45:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab916d8700 1 -- 172.21.15.78:0/1206130460 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c1a0490 con 0x7fab8c102d30 2024-04-25T01:45:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab73fff700 1 -- 172.21.15.78:0/1206130460 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab840073d0 con 0x7fab8c102d30 2024-04-25T01:45:09.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.701+0000 7fab916d8700 1 -- 172.21.15.78:0/1206130460 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c1a0a00 con 0x7fab8c102d30 2024-04-25T01:45:09.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.702+0000 7fab916d8700 1 -- 172.21.15.78:0/1206130460 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8c068f20 con 0x7fab8c102d30 2024-04-25T01:45:09.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.703+0000 7fab73fff700 1 -- 172.21.15.78:0/1206130460 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fab8401c070 con 0x7fab8c102d30 2024-04-25T01:45:09.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.704+0000 7fab73fff700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fab740a2c20 0x7fab740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:09.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.704+0000 7fab8affd700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fab740a2c20 0x7fab740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:09.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.704+0000 7fab73fff700 1 -- 172.21.15.78:0/1206130460 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fab840ce730 con 0x7fab8c102d30 2024-04-25T01:45:09.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.704+0000 7fab8affd700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fab740a2c20 0x7fab740a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fab80006010 tx=0x7fab80009510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:09.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.708+0000 7fab73fff700 1 -- 172.21.15.78:0/1206130460 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fab8409b350 con 0x7fab8c102d30 2024-04-25T01:45:09.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.923+0000 7fab916d8700 1 -- 172.21.15.78:0/1206130460 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fab8c06b010 con 0x7fab740a2c20 2024-04-25T01:45:09.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.924+0000 7fab73fff700 1 -- 172.21.15.78:0/1206130460 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fab8c06b010 con 0x7fab740a2c20 2024-04-25T01:45:09.917 INFO:teuthology.orchestra.run.smithi078.stdout:Backend: cephadm 2024-04-25T01:45:09.918 INFO:teuthology.orchestra.run.smithi078.stdout:Available: Yes 2024-04-25T01:45:09.918 INFO:teuthology.orchestra.run.smithi078.stdout:Paused: No 2024-04-25T01:45:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.926+0000 7fab916d8700 1 -- 172.21.15.78:0/1206130460 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fab740a2c20 msgr2=0x7fab740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.926+0000 7fab916d8700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fab740a2c20 0x7fab740a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fab80006010 tx=0x7fab80009510 comp rx=0 tx=0).stop 2024-04-25T01:45:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.927+0000 7fab916d8700 1 -- 172.21.15.78:0/1206130460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c102d30 msgr2=0x7fab8c19fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.927+0000 7fab916d8700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c102d30 0x7fab8c19fb00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fab8400b9a0 tx=0x7fab8400bcb0 comp rx=0 tx=0).stop 2024-04-25T01:45:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.927+0000 7fab916d8700 1 -- 172.21.15.78:0/1206130460 shutdown_connections 2024-04-25T01:45:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.927+0000 7fab916d8700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fab740a2c20 0x7fab740a50e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.927+0000 7fab916d8700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab8c102d30 0x7fab8c19fb00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.927+0000 7fab916d8700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c101190 0x7fab8c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.927+0000 7fab916d8700 1 --2- 172.21.15.78:0/1206130460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8c102390 0x7fab8c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.927+0000 7fab916d8700 1 -- 172.21.15.78:0/1206130460 >> 172.21.15.78:0/1206130460 conn(0x7fab8c0fc750 msgr2=0x7fab8c06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:09.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.927+0000 7fab916d8700 1 -- 172.21.15.78:0/1206130460 shutdown_connections 2024-04-25T01:45:09.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:09.927+0000 7fab916d8700 1 -- 172.21.15.78:0/1206130460 wait complete. 2024-04-25T01:45:10.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:10 smithi078 ceph-mon[29469]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:45:10.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:10 smithi152 ceph-mon[32465]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:45:10.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:10 smithi196 ceph-mon[32978]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:45:10.666 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T01:45:10.902 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:45:11.292 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:11 smithi078 ceph-mon[29469]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:11.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:11 smithi152 ceph-mon[32465]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:11.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:11 smithi196 ceph-mon[32978]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:12.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:12 smithi152 ceph-mon[32465]: pgmap v349: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:45:12.345 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:12 smithi078 ceph-mon[29469]: pgmap v349: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:45:12.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:12 smithi196 ceph-mon[32978]: pgmap v349: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:45:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.592+0000 7f66db5b0700 1 -- 172.21.15.78:0/1535955227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102d30 msgr2=0x7f66d41095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.592+0000 7f66db5b0700 1 --2- 172.21.15.78:0/1535955227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102d30 0x7f66d41095e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00b0d0 tx=0x7f66cc00b3e0 comp rx=0 tx=0).stop 2024-04-25T01:45:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.592+0000 7f66db5b0700 1 -- 172.21.15.78:0/1535955227 shutdown_connections 2024-04-25T01:45:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.592+0000 7f66db5b0700 1 --2- 172.21.15.78:0/1535955227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66d4101190 0x7f66d41015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.592+0000 7f66db5b0700 1 --2- 172.21.15.78:0/1535955227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102d30 0x7f66d41095e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.592+0000 7f66db5b0700 1 --2- 172.21.15.78:0/1535955227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66d4102390 0x7f66d41027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.592+0000 7f66db5b0700 1 -- 172.21.15.78:0/1535955227 >> 172.21.15.78:0/1535955227 conn(0x7f66d40fc730 msgr2=0x7f66d40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.592+0000 7f66db5b0700 1 -- 172.21.15.78:0/1535955227 shutdown_connections 2024-04-25T01:45:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.592+0000 7f66db5b0700 1 -- 172.21.15.78:0/1535955227 wait complete. 2024-04-25T01:45:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66db5b0700 1 Processor -- start 2024-04-25T01:45:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66db5b0700 1 -- start start 2024-04-25T01:45:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66db5b0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66d4101190 0x7f66d41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66db5b0700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102390 0x7f66d4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66db5b0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66d4102d30 0x7f66d419fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66db5b0700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66d410cdf0 con 0x7f66d4102390 2024-04-25T01:45:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66db5b0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f66d410cc70 con 0x7f66d4101190 2024-04-25T01:45:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66db5b0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66d410cf70 con 0x7f66d4102d30 2024-04-25T01:45:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66d8b4b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102390 0x7f66d4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66d8b4b700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102390 0x7f66d4198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42680/0 (socket says 172.21.15.78:42680) 2024-04-25T01:45:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66d8b4b700 1 -- 172.21.15.78:0/1804746479 learned_addr learned my addr 172.21.15.78:0/1804746479 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66d934c700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66d4101190 0x7f66d41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66d9b4d700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66d4102d30 0x7f66d419fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66d8b4b700 1 -- 172.21.15.78:0/1804746479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66d4102d30 msgr2=0x7f66d419fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66d8b4b700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66d4102d30 0x7f66d419fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66d8b4b700 1 -- 172.21.15.78:0/1804746479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66d4101190 msgr2=0x7f66d41984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66d8b4b700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66d4101190 0x7f66d41984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66d8b4b700 1 -- 172.21.15.78:0/1804746479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d41a0130 con 0x7f66d4102390 2024-04-25T01:45:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66d934c700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66d4101190 0x7f66d41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66d9b4d700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66d4102d30 0x7f66d419fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.593+0000 7f66d8b4b700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102390 0x7f66d4198a10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c9e0 tx=0x7f66d000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.594+0000 7f66ca7fc700 1 -- 172.21.15.78:0/1804746479 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0007ac0 con 0x7f66d4102390 2024-04-25T01:45:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.594+0000 7f66ca7fc700 1 -- 172.21.15.78:0/1804746479 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66d0007c60 con 0x7f66d4102390 2024-04-25T01:45:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.594+0000 7f66db5b0700 1 -- 172.21.15.78:0/1804746479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d41a0420 con 0x7f66d4102390 2024-04-25T01:45:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.594+0000 7f66ca7fc700 1 -- 172.21.15.78:0/1804746479 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0018710 con 0x7f66d4102390 2024-04-25T01:45:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.594+0000 7f66db5b0700 1 -- 172.21.15.78:0/1804746479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d41a09e0 con 0x7f66d4102390 2024-04-25T01:45:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.595+0000 7f66db5b0700 1 -- 172.21.15.78:0/1804746479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d4068f20 con 0x7f66d4102390 2024-04-25T01:45:12.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.596+0000 7f66ca7fc700 1 -- 172.21.15.78:0/1804746479 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f66d00188d0 con 0x7f66d4102390 2024-04-25T01:45:12.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.596+0000 7f66ca7fc700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f66c00a2c20 0x7f66c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:12.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.596+0000 7f66d934c700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f66c00a2c20 0x7f66c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:12.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.597+0000 7f66ca7fc700 1 -- 172.21.15.78:0/1804746479 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f66d00cf590 con 0x7f66d4102390 2024-04-25T01:45:12.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.597+0000 7f66d934c700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f66c00a2c20 0x7f66c00a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f66d41021f0 tx=0x7f66c4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:12.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.600+0000 7f66ca7fc700 1 -- 172.21.15.78:0/1804746479 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f66d0014030 con 0x7f66d4102390 2024-04-25T01:45:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.806+0000 7f66db5b0700 1 -- 172.21.15.78:0/1804746479 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f66d40fe840 con 0x7f66c00a2c20 2024-04-25T01:45:12.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.819+0000 7f66ca7fc700 1 -- 172.21.15.78:0/1804746479 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f66d40fe840 con 0x7f66c00a2c20 2024-04-25T01:45:12.812 INFO:teuthology.orchestra.run.smithi078.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:45:12.812 INFO:teuthology.orchestra.run.smithi078.stdout:mds.a smithi078 running (2m) 24s ago 2m 17.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef b1e800d9d8d9 2024-04-25T01:45:12.812 INFO:teuthology.orchestra.run.smithi078.stdout:mds.b smithi152 running (2m) 54s ago 2m 25.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 87728433819d 2024-04-25T01:45:12.812 INFO:teuthology.orchestra.run.smithi078.stdout:mds.c smithi196 running (2m) 54s ago 2m 19.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 35beadf15ca7 2024-04-25T01:45:12.812 INFO:teuthology.orchestra.run.smithi078.stdout:mds.d smithi078 running (2m) 24s ago 2m 13.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef 81055bfaff02 2024-04-25T01:45:12.813 INFO:teuthology.orchestra.run.smithi078.stdout:mds.e smithi152 running (2m) 54s ago 2m 15.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef ebc5baaea7ef 2024-04-25T01:45:12.813 INFO:teuthology.orchestra.run.smithi078.stdout:mds.f smithi196 running (2m) 54s ago 2m 16.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef 0bf5b3311815 2024-04-25T01:45:12.813 INFO:teuthology.orchestra.run.smithi078.stdout:mds.g smithi078 running (2m) 24s ago 2m 15.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef e27d76e841e0 2024-04-25T01:45:12.813 INFO:teuthology.orchestra.run.smithi078.stdout:mds.h smithi152 running (2m) 54s ago 2m 19.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef b6b3e0c9d0ee 2024-04-25T01:45:12.813 INFO:teuthology.orchestra.run.smithi078.stdout:mds.i smithi196 running (2m) 54s ago 2m 19.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef 3241e6bf050a 2024-04-25T01:45:12.813 INFO:teuthology.orchestra.run.smithi078.stdout:mds.j smithi078 running (2m) 24s ago 2m 15.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef 9a1216143646 2024-04-25T01:45:12.813 INFO:teuthology.orchestra.run.smithi078.stdout:mds.k smithi152 running (2m) 54s ago 2m 17.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7d32f90d7043 2024-04-25T01:45:12.813 INFO:teuthology.orchestra.run.smithi078.stdout:mds.l smithi196 running (2m) 54s ago 2m 17.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef e124f5eb5ddf 2024-04-25T01:45:12.813 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.x smithi078 *:9283,8765 running (9m) 24s ago 9m 515M - 19.0.0-2445-gd2c45d97 f68e37b807ef 20678b9d90e1 2024-04-25T01:45:12.813 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.y smithi152 *:8443,8765 running (7m) 54s ago 7m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef 270917cafea2 2024-04-25T01:45:12.813 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.z smithi196 *:8443,8765 running (7m) 54s ago 7m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef ab555bd66c8c 2024-04-25T01:45:12.813 INFO:teuthology.orchestra.run.smithi078.stdout:mon.a smithi078 running (9m) 24s ago 9m 51.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef c83da7f37b2e 2024-04-25T01:45:12.813 INFO:teuthology.orchestra.run.smithi078.stdout:mon.b smithi152 running (8m) 54s ago 8m 39.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 5608e8e66536 2024-04-25T01:45:12.814 INFO:teuthology.orchestra.run.smithi078.stdout:mon.c smithi196 running (8m) 54s ago 8m 41.1M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 7fe86deb7564 2024-04-25T01:45:12.814 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 smithi078 running (7m) 24s ago 7m 46.2M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 0fa2caa4a2ca 2024-04-25T01:45:12.814 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 smithi152 running (7m) 54s ago 7m 47.0M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 6f5b16a92481 2024-04-25T01:45:12.814 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 smithi196 running (6m) 54s ago 6m 50.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 6c3c03ecd59b 2024-04-25T01:45:12.814 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 smithi078 running (6m) 24s ago 6m 44.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 2432217ecad5 2024-04-25T01:45:12.814 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 smithi152 running (5m) 54s ago 5m 55.1M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 795b01241629 2024-04-25T01:45:12.814 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 smithi196 running (5m) 54s ago 5m 56.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 85f1045d5a1d 2024-04-25T01:45:12.814 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 smithi078 running (5m) 24s ago 5m 56.3M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 160409d9129f 2024-04-25T01:45:12.814 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 smithi152 running (4m) 54s ago 4m 43.9M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef cd4a667653b0 2024-04-25T01:45:12.814 INFO:teuthology.orchestra.run.smithi078.stdout:osd.8 smithi196 running (4m) 54s ago 4m 53.9M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef dffdae8b1b29 2024-04-25T01:45:12.814 INFO:teuthology.orchestra.run.smithi078.stdout:osd.9 smithi078 running (3m) 24s ago 3m 52.9M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 0ef4fe01cdee 2024-04-25T01:45:12.815 INFO:teuthology.orchestra.run.smithi078.stdout:osd.10 smithi152 running (3m) 54s ago 3m 53.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 123cf9b53c0a 2024-04-25T01:45:12.815 INFO:teuthology.orchestra.run.smithi078.stdout:osd.11 smithi196 running (2m) 54s ago 2m 44.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 7c4629808169 2024-04-25T01:45:12.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.821+0000 7f66db5b0700 1 -- 172.21.15.78:0/1804746479 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f66c00a2c20 msgr2=0x7f66c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:12.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.821+0000 7f66db5b0700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f66c00a2c20 0x7f66c00a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f66d41021f0 tx=0x7f66c4008040 comp rx=0 tx=0).stop 2024-04-25T01:45:12.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.821+0000 7f66db5b0700 1 -- 172.21.15.78:0/1804746479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102390 msgr2=0x7f66d4198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:12.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.821+0000 7f66db5b0700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102390 0x7f66d4198a10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c9e0 tx=0x7f66d000cda0 comp rx=0 tx=0).stop 2024-04-25T01:45:12.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.822+0000 7f66db5b0700 1 -- 172.21.15.78:0/1804746479 shutdown_connections 2024-04-25T01:45:12.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.822+0000 7f66db5b0700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f66c00a2c20 0x7f66c00a50e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:12.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.822+0000 7f66db5b0700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66d4102d30 0x7f66d419fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:12.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.822+0000 7f66db5b0700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f66d4101190 0x7f66d41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:12.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.822+0000 7f66db5b0700 1 --2- 172.21.15.78:0/1804746479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66d4102390 0x7f66d4198a10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:12.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.822+0000 7f66db5b0700 1 -- 172.21.15.78:0/1804746479 >> 172.21.15.78:0/1804746479 conn(0x7f66d40fc730 msgr2=0x7f66d40fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:12.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.822+0000 7f66db5b0700 1 -- 172.21.15.78:0/1804746479 shutdown_connections 2024-04-25T01:45:12.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:12.822+0000 7f66db5b0700 1 -- 172.21.15.78:0/1804746479 wait complete. 2024-04-25T01:45:13.643 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T01:45:13.758 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:13 smithi078 ceph-mon[29469]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:13.758 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:13 smithi078 ceph-mon[29469]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:45:13.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:13 smithi152 ceph-mon[32465]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:13.799 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:13 smithi152 ceph-mon[32465]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:45:13.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:13 smithi196 ceph-mon[32978]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:13.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:13 smithi196 ceph-mon[32978]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:45:13.880 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:45:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.679+0000 7f55ebc1d700 1 -- 172.21.15.78:0/2372538289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55e410cc90 msgr2=0x7f55e410d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.679+0000 7f55ebc1d700 1 --2- 172.21.15.78:0/2372538289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55e410cc90 0x7f55e410d0b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f55e00099c0 tx=0x7f55e0009cd0 comp rx=0 tx=0).stop 2024-04-25T01:45:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.679+0000 7f55ebc1d700 1 -- 172.21.15.78:0/2372538289 shutdown_connections 2024-04-25T01:45:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.679+0000 7f55ebc1d700 1 --2- 172.21.15.78:0/2372538289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55e410e830 0x7f55e41150e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.679+0000 7f55ebc1d700 1 --2- 172.21.15.78:0/2372538289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55e410cc90 0x7f55e410d0b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.679+0000 7f55ebc1d700 1 --2- 172.21.15.78:0/2372538289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e410de90 0x7f55e410e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.679+0000 7f55ebc1d700 1 -- 172.21.15.78:0/2372538289 >> 172.21.15.78:0/2372538289 conn(0x7f55e406cdd0 msgr2=0x7f55e406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.679+0000 7f55ebc1d700 1 -- 172.21.15.78:0/2372538289 shutdown_connections 2024-04-25T01:45:15.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.679+0000 7f55ebc1d700 1 -- 172.21.15.78:0/2372538289 wait complete. 2024-04-25T01:45:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55ebc1d700 1 Processor -- start 2024-04-25T01:45:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55ebc1d700 1 -- start start 2024-04-25T01:45:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55ebc1d700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55e410cc90 0x7f55e41a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55ebc1d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55e410de90 0x7f55e41a3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55ebc1d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e410e830 0x7f55e41aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55ebc1d700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55e4118950 con 0x7f55e410cc90 2024-04-25T01:45:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55ebc1d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f55e41187d0 con 0x7f55e410e830 2024-04-25T01:45:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55ebc1d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55e4118ad0 con 0x7f55e410de90 2024-04-25T01:45:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55e99b9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55e410cc90 0x7f55e41a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55e99b9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55e410cc90 0x7f55e41a3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42684/0 (socket says 172.21.15.78:42684) 2024-04-25T01:45:15.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55e99b9700 1 -- 172.21.15.78:0/4226761750 learned_addr learned my addr 172.21.15.78:0/4226761750 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55e91b8700 1 --2- 172.21.15.78:0/4226761750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55e410de90 0x7f55e41a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55e99b9700 1 -- 172.21.15.78:0/4226761750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55e410de90 msgr2=0x7f55e41a3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55e99b9700 1 --2- 172.21.15.78:0/4226761750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55e410de90 0x7f55e41a3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55e99b9700 1 -- 172.21.15.78:0/4226761750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e410e830 msgr2=0x7f55e41aa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55e99b9700 1 --2- 172.21.15.78:0/4226761750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e410e830 0x7f55e41aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55e99b9700 1 -- 172.21.15.78:0/4226761750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e41aac80 con 0x7f55e410cc90 2024-04-25T01:45:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.680+0000 7f55e99b9700 1 --2- 172.21.15.78:0/4226761750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55e410cc90 0x7f55e41a3020 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f55e00057c0 tx=0x7f55e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.681+0000 7f55daffd700 1 -- 172.21.15.78:0/4226761750 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0027070 con 0x7f55e410cc90 2024-04-25T01:45:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.681+0000 7f55daffd700 1 -- 172.21.15.78:0/4226761750 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e00058e0 con 0x7f55e410cc90 2024-04-25T01:45:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.681+0000 7f55ebc1d700 1 -- 172.21.15.78:0/4226761750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e41aaf10 con 0x7f55e410cc90 2024-04-25T01:45:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.681+0000 7f55daffd700 1 -- 172.21.15.78:0/4226761750 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e000f460 con 0x7f55e410cc90 2024-04-25T01:45:15.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.681+0000 7f55ebc1d700 1 -- 172.21.15.78:0/4226761750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e41ab3e0 con 0x7f55e410cc90 2024-04-25T01:45:15.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.682+0000 7f55ebc1d700 1 -- 172.21.15.78:0/4226761750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e4068f20 con 0x7f55e410cc90 2024-04-25T01:45:15.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.683+0000 7f55daffd700 1 -- 172.21.15.78:0/4226761750 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f55e002a070 con 0x7f55e410cc90 2024-04-25T01:45:15.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.683+0000 7f55daffd700 1 --2- 172.21.15.78:0/4226761750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f55d00a2c20 0x7f55d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:15.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.683+0000 7f55e91b8700 1 --2- 172.21.15.78:0/4226761750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f55d00a2c20 0x7f55d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:15.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.684+0000 7f55daffd700 1 -- 172.21.15.78:0/4226761750 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f55e00d0aa0 con 0x7f55e410cc90 2024-04-25T01:45:15.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.684+0000 7f55e91b8700 1 --2- 172.21.15.78:0/4226761750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f55d00a2c20 0x7f55d00a50e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f55d4006fd0 tx=0x7f55d4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:15.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.687+0000 7f55daffd700 1 -- 172.21.15.78:0/4226761750 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f55e0031020 con 0x7f55e410cc90 2024-04-25T01:45:15.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.892+0000 7f55ebc1d700 1 -- 172.21.15.78:0/4226761750 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f55e4062670 con 0x7f55d00a2c20 2024-04-25T01:45:15.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.898+0000 7f55daffd700 1 -- 172.21.15.78:0/4226761750 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f55e4062670 con 0x7f55d00a2c20 2024-04-25T01:45:15.891 INFO:teuthology.orchestra.run.smithi078.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T01:45:15.891 INFO:teuthology.orchestra.run.smithi078.stdout:mds.all 12/12 57s ago 2m smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi152=b;smithi152=e;smithi152=h;smithi152=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-04-25T01:45:15.892 INFO:teuthology.orchestra.run.smithi078.stdout:mgr 3/3 57s ago 7m smithi078=x;smithi152=y;smithi196=z;count:3 2024-04-25T01:45:15.892 INFO:teuthology.orchestra.run.smithi078.stdout:mon 3/3 57s ago 8m smithi078:172.21.15.78=a;smithi152:172.21.15.152=b;smithi196:172.21.15.196=c;count:3 2024-04-25T01:45:15.892 INFO:teuthology.orchestra.run.smithi078.stdout:osd 12 57s ago - 2024-04-25T01:45:15.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.900+0000 7f55ebc1d700 1 -- 172.21.15.78:0/4226761750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f55d00a2c20 msgr2=0x7f55d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:15.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.900+0000 7f55ebc1d700 1 --2- 172.21.15.78:0/4226761750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f55d00a2c20 0x7f55d00a50e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f55d4006fd0 tx=0x7f55d4008040 comp rx=0 tx=0).stop 2024-04-25T01:45:15.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.900+0000 7f55ebc1d700 1 -- 172.21.15.78:0/4226761750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55e410cc90 msgr2=0x7f55e41a3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:15.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.900+0000 7f55ebc1d700 1 --2- 172.21.15.78:0/4226761750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55e410cc90 0x7f55e41a3020 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f55e00057c0 tx=0x7f55e00057f0 comp rx=0 tx=0).stop 2024-04-25T01:45:15.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.900+0000 7f55ebc1d700 1 -- 172.21.15.78:0/4226761750 shutdown_connections 2024-04-25T01:45:15.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.900+0000 7f55ebc1d700 1 --2- 172.21.15.78:0/4226761750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f55d00a2c20 0x7f55d00a50e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:15.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.900+0000 7f55ebc1d700 1 --2- 172.21.15.78:0/4226761750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e410e830 0x7f55e41aa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:15.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.900+0000 7f55ebc1d700 1 --2- 172.21.15.78:0/4226761750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55e410cc90 0x7f55e41a3020 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:15.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.900+0000 7f55ebc1d700 1 --2- 172.21.15.78:0/4226761750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55e410de90 0x7f55e41a3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:15.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.900+0000 7f55ebc1d700 1 -- 172.21.15.78:0/4226761750 >> 172.21.15.78:0/4226761750 conn(0x7f55e406cdd0 msgr2=0x7f55e4111260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:15.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.901+0000 7f55ebc1d700 1 -- 172.21.15.78:0/4226761750 shutdown_connections 2024-04-25T01:45:15.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:15.901+0000 7f55ebc1d700 1 -- 172.21.15.78:0/4226761750 wait complete. 2024-04-25T01:45:16.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:15 smithi078 ceph-mon[29469]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:45:16.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:15 smithi152 ceph-mon[32465]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:45:16.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:15 smithi196 ceph-mon[32978]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:45:16.783 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T01:45:17.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:16 smithi078 ceph-mon[29469]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:17.036 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:45:17.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:16 smithi152 ceph-mon[32465]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:17.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:16 smithi196 ceph-mon[32978]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:18.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:17 smithi078 ceph-mon[29469]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:45:18.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T01:45:18.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T01:45:18.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-25T01:45:18.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]: dispatch 2024-04-25T01:45:18.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:17 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T01:45:18.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:17 smithi152 ceph-mon[32465]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:45:18.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T01:45:18.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T01:45:18.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-25T01:45:18.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]: dispatch 2024-04-25T01:45:18.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:17 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T01:45:18.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:18 smithi196 ceph-mon[32978]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:45:18.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T01:45:18.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T01:45:18.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-25T01:45:18.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]: dispatch 2024-04-25T01:45:18.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:18 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T01:45:18.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.853+0000 7f2ac8021700 1 -- 172.21.15.78:0/151873170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0102d30 msgr2=0x7f2ac01095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:18.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.853+0000 7f2ac8021700 1 --2- 172.21.15.78:0/151873170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0102d30 0x7f2ac01095e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800b0d0 tx=0x7f2ab800b3e0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.853+0000 7f2ac8021700 1 -- 172.21.15.78:0/151873170 shutdown_connections 2024-04-25T01:45:18.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.853+0000 7f2ac8021700 1 --2- 172.21.15.78:0/151873170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0102d30 0x7f2ac01095e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.853+0000 7f2ac8021700 1 --2- 172.21.15.78:0/151873170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac0101190 0x7f2ac01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.853+0000 7f2ac8021700 1 --2- 172.21.15.78:0/151873170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac0102390 0x7f2ac01027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.853+0000 7f2ac8021700 1 -- 172.21.15.78:0/151873170 >> 172.21.15.78:0/151873170 conn(0x7f2ac00fc730 msgr2=0x7f2ac00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:18.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.853+0000 7f2ac8021700 1 -- 172.21.15.78:0/151873170 shutdown_connections 2024-04-25T01:45:18.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.853+0000 7f2ac8021700 1 -- 172.21.15.78:0/151873170 wait complete. 2024-04-25T01:45:18.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac8021700 1 Processor -- start 2024-04-25T01:45:18.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac8021700 1 -- start start 2024-04-25T01:45:18.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac8021700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0101190 0x7f2ac0198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:18.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac8021700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac0102390 0x7f2ac0198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:18.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac8021700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac0102d30 0x7f2ac019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:18.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac8021700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac010ce70 con 0x7f2ac0101190 2024-04-25T01:45:18.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac8021700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac010ccf0 con 0x7f2ac0102d30 2024-04-25T01:45:18.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac8021700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac010cff0 con 0x7f2ac0102390 2024-04-25T01:45:18.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac5dbd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0101190 0x7f2ac0198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:18.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac5dbd700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0101190 0x7f2ac0198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42714/0 (socket says 172.21.15.78:42714) 2024-04-25T01:45:18.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac5dbd700 1 -- 172.21.15.78:0/2087690287 learned_addr learned my addr 172.21.15.78:0/2087690287 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:18.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac55bc700 1 --2- 172.21.15.78:0/2087690287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac0102390 0x7f2ac0198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:18.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac65be700 1 --2- 172.21.15.78:0/2087690287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac0102d30 0x7f2ac019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:18.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac5dbd700 1 -- 172.21.15.78:0/2087690287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac0102390 msgr2=0x7f2ac0198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:18.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac5dbd700 1 --2- 172.21.15.78:0/2087690287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac0102390 0x7f2ac0198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac5dbd700 1 -- 172.21.15.78:0/2087690287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac0102d30 msgr2=0x7f2ac019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:18.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac5dbd700 1 --2- 172.21.15.78:0/2087690287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac0102d30 0x7f2ac019fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.854+0000 7f2ac5dbd700 1 -- 172.21.15.78:0/2087690287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac01a0260 con 0x7f2ac0101190 2024-04-25T01:45:18.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.855+0000 7f2ac5dbd700 1 --2- 172.21.15.78:0/2087690287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0101190 0x7f2ac0198600 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00b530 tx=0x7f2abc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:18.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.855+0000 7f2ab6ffd700 1 -- 172.21.15.78:0/2087690287 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc015020 con 0x7f2ac0101190 2024-04-25T01:45:18.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.855+0000 7f2ac8021700 1 -- 172.21.15.78:0/2087690287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac01a0550 con 0x7f2ac0101190 2024-04-25T01:45:18.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.855+0000 7f2ab6ffd700 1 -- 172.21.15.78:0/2087690287 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2abc011d10 con 0x7f2ac0101190 2024-04-25T01:45:18.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.855+0000 7f2ab6ffd700 1 -- 172.21.15.78:0/2087690287 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc00f2e0 con 0x7f2ac0101190 2024-04-25T01:45:18.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.855+0000 7f2ac8021700 1 -- 172.21.15.78:0/2087690287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac01a0b10 con 0x7f2ac0101190 2024-04-25T01:45:18.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.856+0000 7f2ac8021700 1 -- 172.21.15.78:0/2087690287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac0068f20 con 0x7f2ac0101190 2024-04-25T01:45:18.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.857+0000 7f2ab6ffd700 1 -- 172.21.15.78:0/2087690287 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2abc018020 con 0x7f2ac0101190 2024-04-25T01:45:18.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.857+0000 7f2ab6ffd700 1 --2- 172.21.15.78:0/2087690287 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2aac0a2c20 0x7f2aac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:18.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.858+0000 7f2ac55bc700 1 --2- 172.21.15.78:0/2087690287 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2aac0a2c20 0x7f2aac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:18.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.858+0000 7f2ab6ffd700 1 -- 172.21.15.78:0/2087690287 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9224+0+0 (secure 0 0 0) 0x7f2abc0cefb0 con 0x7f2ac0101190 2024-04-25T01:45:18.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.858+0000 7f2ac55bc700 1 --2- 172.21.15.78:0/2087690287 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2aac0a2c20 0x7f2aac0a50e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009a10 tx=0x7f2ab0009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:18.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:18.861+0000 7f2ab6ffd700 1 -- 172.21.15.78:0/2087690287 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2abc01f020 con 0x7f2ac0101190 2024-04-25T01:45:19.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:19.071+0000 7f2ac8021700 1 -- 172.21.15.78:0/2087690287 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f2ac006a620 con 0x7f2aac0a2c20 2024-04-25T01:45:19.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:19.073+0000 7f2ab6ffd700 1 -- 172.21.15.78:0/2087690287 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f2ac006a620 con 0x7f2aac0a2c20 2024-04-25T01:45:19.065 INFO:teuthology.orchestra.run.smithi078.stdout:HOST ADDR LABELS STATUS 2024-04-25T01:45:19.065 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 172.21.15.78 2024-04-25T01:45:19.065 INFO:teuthology.orchestra.run.smithi078.stdout:smithi152 172.21.15.152 2024-04-25T01:45:19.065 INFO:teuthology.orchestra.run.smithi078.stdout:smithi196 172.21.15.196 2024-04-25T01:45:19.066 INFO:teuthology.orchestra.run.smithi078.stdout:3 hosts in cluster 2024-04-25T01:45:19.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:19.075+0000 7f2ac8021700 1 -- 172.21.15.78:0/2087690287 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2aac0a2c20 msgr2=0x7f2aac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:19.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:19.075+0000 7f2ac8021700 1 --2- 172.21.15.78:0/2087690287 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2aac0a2c20 0x7f2aac0a50e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009a10 tx=0x7f2ab0009190 comp rx=0 tx=0).stop 2024-04-25T01:45:19.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:19.075+0000 7f2ac8021700 1 -- 172.21.15.78:0/2087690287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0101190 msgr2=0x7f2ac0198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:19.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:19.075+0000 7f2ac8021700 1 --2- 172.21.15.78:0/2087690287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0101190 0x7f2ac0198600 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00b530 tx=0x7f2abc00b840 comp rx=0 tx=0).stop 2024-04-25T01:45:19.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:19.075+0000 7f2ac8021700 1 -- 172.21.15.78:0/2087690287 shutdown_connections 2024-04-25T01:45:19.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:19.075+0000 7f2ac8021700 1 --2- 172.21.15.78:0/2087690287 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2aac0a2c20 0x7f2aac0a50e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:19.075+0000 7f2ac8021700 1 --2- 172.21.15.78:0/2087690287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ac0102d30 0x7f2ac019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:19.075+0000 7f2ac8021700 1 --2- 172.21.15.78:0/2087690287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ac0101190 0x7f2ac0198600 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:19.075+0000 7f2ac8021700 1 --2- 172.21.15.78:0/2087690287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac0102390 0x7f2ac0198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:19.075+0000 7f2ac8021700 1 -- 172.21.15.78:0/2087690287 >> 172.21.15.78:0/2087690287 conn(0x7f2ac00fc730 msgr2=0x7f2ac0069f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:19.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:19.075+0000 7f2ac8021700 1 -- 172.21.15.78:0/2087690287 shutdown_connections 2024-04-25T01:45:19.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:19.075+0000 7f2ac8021700 1 -- 172.21.15.78:0/2087690287 wait complete. 2024-04-25T01:45:19.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T01:45:19.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T01:45:19.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-25T01:45:19.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]': finished 2024-04-25T01:45:19.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:19 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T01:45:19.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:19 smithi078 ceph-mon[29469]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T01:45:19.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:19 smithi078 ceph-mon[29469]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T01:45:19.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T01:45:19.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T01:45:19.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-25T01:45:19.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]': finished 2024-04-25T01:45:19.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:19 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T01:45:19.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:19 smithi152 ceph-mon[32465]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T01:45:19.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:19 smithi152 ceph-mon[32465]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T01:45:19.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T01:45:19.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T01:45:19.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-25T01:45:19.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1, 7, 11]}]': finished 2024-04-25T01:45:19.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:19 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T01:45:19.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:19 smithi196 ceph-mon[32978]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T01:45:19.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:19 smithi196 ceph-mon[32978]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T01:45:19.791 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T01:45:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:45:20.151 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:20 smithi078 ceph-mon[29469]: pgmap v354: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T01:45:20.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:20 smithi152 ceph-mon[32465]: pgmap v354: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T01:45:20.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:20 smithi196 ceph-mon[32978]: pgmap v354: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T01:45:21.240 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:21 smithi078 ceph-mon[29469]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:21.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:21 smithi152 ceph-mon[32465]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:21.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:21 smithi196 ceph-mon[32978]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:21.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.789+0000 7fb2d6911700 1 -- 172.21.15.78:0/229185378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2d0103e60 msgr2=0x7fb2d010a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:21.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.789+0000 7fb2d6911700 1 --2- 172.21.15.78:0/229185378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2d0103e60 0x7fb2d010a710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800b0d0 tx=0x7fb2c800b3e0 comp rx=0 tx=0).stop 2024-04-25T01:45:21.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.789+0000 7fb2d6911700 1 -- 172.21.15.78:0/229185378 shutdown_connections 2024-04-25T01:45:21.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.789+0000 7fb2d6911700 1 --2- 172.21.15.78:0/229185378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2d01022c0 0x7fb2d01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:21.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.789+0000 7fb2d6911700 1 --2- 172.21.15.78:0/229185378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2d0103e60 0x7fb2d010a710 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:21.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.789+0000 7fb2d6911700 1 --2- 172.21.15.78:0/229185378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2d01034c0 0x7fb2d0103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:21.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.789+0000 7fb2d6911700 1 -- 172.21.15.78:0/229185378 >> 172.21.15.78:0/229185378 conn(0x7fb2d00fd880 msgr2=0x7fb2d00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:21.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.789+0000 7fb2d6911700 1 -- 172.21.15.78:0/229185378 shutdown_connections 2024-04-25T01:45:21.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.789+0000 7fb2d6911700 1 -- 172.21.15.78:0/229185378 wait complete. 2024-04-25T01:45:21.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.790+0000 7fb2d6911700 1 Processor -- start 2024-04-25T01:45:21.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.790+0000 7fb2d6911700 1 -- start start 2024-04-25T01:45:21.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.790+0000 7fb2d6911700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2d01022c0 0x7fb2d01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:21.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.790+0000 7fb2d6911700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2d01034c0 0x7fb2d01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:21.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.790+0000 7fb2d6911700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2d0103e60 0x7fb2d019f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:21.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.790+0000 7fb2d6911700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d010de80 con 0x7fb2d0103e60 2024-04-25T01:45:21.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.790+0000 7fb2d6911700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d010dd00 con 0x7fb2d01034c0 2024-04-25T01:45:21.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.790+0000 7fb2d6911700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d010e000 con 0x7fb2d01022c0 2024-04-25T01:45:21.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.790+0000 7fb2d4eae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2d0103e60 0x7fb2d019f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:21.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.790+0000 7fb2d4eae700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2d0103e60 0x7fb2d019f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42770/0 (socket says 172.21.15.78:42770) 2024-04-25T01:45:21.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.790+0000 7fb2d4eae700 1 -- 172.21.15.78:0/1305785742 learned_addr learned my addr 172.21.15.78:0/1305785742 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:21.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.791+0000 7fb2cf7fe700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2d01034c0 0x7fb2d01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:21.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.791+0000 7fb2cffff700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2d01022c0 0x7fb2d01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:21.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.791+0000 7fb2d4eae700 1 -- 172.21.15.78:0/1305785742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2d01022c0 msgr2=0x7fb2d01983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:21.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.791+0000 7fb2d4eae700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2d01022c0 0x7fb2d01983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:21.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.791+0000 7fb2d4eae700 1 -- 172.21.15.78:0/1305785742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2d01034c0 msgr2=0x7fb2d01988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:21.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.791+0000 7fb2d4eae700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2d01034c0 0x7fb2d01988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:21.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.791+0000 7fb2d4eae700 1 -- 172.21.15.78:0/1305785742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d01a0000 con 0x7fb2d0103e60 2024-04-25T01:45:21.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.791+0000 7fb2cffff700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2d01022c0 0x7fb2d01983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:21.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.791+0000 7fb2d4eae700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2d0103e60 0x7fb2d019f960 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8007d10 tx=0x7fb2c8007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:21.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.791+0000 7fb2cd7fa700 1 -- 172.21.15.78:0/1305785742 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8021020 con 0x7fb2d0103e60 2024-04-25T01:45:21.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.791+0000 7fb2cd7fa700 1 -- 172.21.15.78:0/1305785742 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2c801d950 con 0x7fb2d0103e60 2024-04-25T01:45:21.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.791+0000 7fb2d6911700 1 -- 172.21.15.78:0/1305785742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d01a0290 con 0x7fb2d0103e60 2024-04-25T01:45:21.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.791+0000 7fb2cd7fa700 1 -- 172.21.15.78:0/1305785742 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8005450 con 0x7fb2d0103e60 2024-04-25T01:45:21.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.791+0000 7fb2d6911700 1 -- 172.21.15.78:0/1305785742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d01a07d0 con 0x7fb2d0103e60 2024-04-25T01:45:21.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.792+0000 7fb2d6911700 1 -- 172.21.15.78:0/1305785742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d0068f20 con 0x7fb2d0103e60 2024-04-25T01:45:21.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.794+0000 7fb2cd7fa700 1 -- 172.21.15.78:0/1305785742 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb2c8024020 con 0x7fb2d0103e60 2024-04-25T01:45:21.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.795+0000 7fb2cd7fa700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb2b80a2c20 0x7fb2b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:21.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.795+0000 7fb2cffff700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb2b80a2c20 0x7fb2b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:21.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.795+0000 7fb2cd7fa700 1 -- 172.21.15.78:0/1305785742 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fb2c800f020 con 0x7fb2d0103e60 2024-04-25T01:45:21.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.795+0000 7fb2cffff700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb2b80a2c20 0x7fb2b80a50e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0103320 tx=0x7fb2c400b670 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:21.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:21.798+0000 7fb2cd7fa700 1 -- 172.21.15.78:0/1305785742 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb2c809da00 con 0x7fb2d0103e60 2024-04-25T01:45:21.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:22.006+0000 7fb2d6911700 1 -- 172.21.15.78:0/1305785742 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb2d0062670 con 0x7fb2b80a2c20 2024-04-25T01:45:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:22.008+0000 7fb2cd7fa700 1 -- 172.21.15.78:0/1305785742 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fb2d0062670 con 0x7fb2b80a2c20 2024-04-25T01:45:22.000 INFO:teuthology.orchestra.run.smithi078.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T01:45:22.001 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T01:45:22.001 INFO:teuthology.orchestra.run.smithi078.stdout:smithi152 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JT400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T01:45:22.001 INFO:teuthology.orchestra.run.smithi078.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T01:45:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:22.010+0000 7fb2d6911700 1 -- 172.21.15.78:0/1305785742 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb2b80a2c20 msgr2=0x7fb2b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:22.010+0000 7fb2d6911700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb2b80a2c20 0x7fb2b80a50e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0103320 tx=0x7fb2c400b670 comp rx=0 tx=0).stop 2024-04-25T01:45:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:22.010+0000 7fb2d6911700 1 -- 172.21.15.78:0/1305785742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2d0103e60 msgr2=0x7fb2d019f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:22.010+0000 7fb2d6911700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2d0103e60 0x7fb2d019f960 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8007d10 tx=0x7fb2c8007d40 comp rx=0 tx=0).stop 2024-04-25T01:45:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:22.011+0000 7fb2d6911700 1 -- 172.21.15.78:0/1305785742 shutdown_connections 2024-04-25T01:45:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:22.011+0000 7fb2d6911700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb2b80a2c20 0x7fb2b80a50e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:22.011+0000 7fb2d6911700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2d01022c0 0x7fb2d01983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:22.011+0000 7fb2d6911700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2d0103e60 0x7fb2d019f960 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:22.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:22.011+0000 7fb2d6911700 1 --2- 172.21.15.78:0/1305785742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2d01034c0 0x7fb2d01988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:22.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:22.011+0000 7fb2d6911700 1 -- 172.21.15.78:0/1305785742 >> 172.21.15.78:0/1305785742 conn(0x7fb2d00fd880 msgr2=0x7fb2d01066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:22.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:22.011+0000 7fb2d6911700 1 -- 172.21.15.78:0/1305785742 shutdown_connections 2024-04-25T01:45:22.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:22.011+0000 7fb2d6911700 1 -- 172.21.15.78:0/1305785742 wait complete. 2024-04-25T01:45:22.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:22 smithi078 ceph-mon[29469]: pgmap v356: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:45:22.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:22 smithi152 ceph-mon[32465]: pgmap v356: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:45:22.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:22 smithi196 ceph-mon[32978]: pgmap v356: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T01:45:22.720 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T01:45:22.729 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T01:45:22.729 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T01:45:22.975 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:45:23.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:23 smithi078 ceph-mon[29469]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:23.299 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:23 smithi152 ceph-mon[32465]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:23.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:23 smithi196 ceph-mon[32978]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:24.160 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:24 smithi078 ceph-mon[29469]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s; 7.8 KiB/s, 2 objects/s recovering 2024-04-25T01:45:24.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:24 smithi152 ceph-mon[32465]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s; 7.8 KiB/s, 2 objects/s recovering 2024-04-25T01:45:24.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:24 smithi196 ceph-mon[32978]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s; 7.8 KiB/s, 2 objects/s recovering 2024-04-25T01:45:24.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.714+0000 7fed5ddf9700 1 -- 172.21.15.78:0/1651693113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed58100550 msgr2=0x7fed58100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:24.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.714+0000 7fed5ddf9700 1 --2- 172.21.15.78:0/1651693113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed58100550 0x7fed58100970 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fed48009990 tx=0x7fed48009ca0 comp rx=0 tx=0).stop 2024-04-25T01:45:24.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.714+0000 7fed5ddf9700 1 -- 172.21.15.78:0/1651693113 shutdown_connections 2024-04-25T01:45:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.714+0000 7fed5ddf9700 1 --2- 172.21.15.78:0/1651693113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed58103ed0 0x7fed5810a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.714+0000 7fed5ddf9700 1 --2- 172.21.15.78:0/1651693113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed58100550 0x7fed58100970 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.714+0000 7fed5ddf9700 1 --2- 172.21.15.78:0/1651693113 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed58103530 0x7fed58103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.714+0000 7fed5ddf9700 1 -- 172.21.15.78:0/1651693113 >> 172.21.15.78:0/1651693113 conn(0x7fed580fbe90 msgr2=0x7fed580fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.714+0000 7fed5ddf9700 1 -- 172.21.15.78:0/1651693113 shutdown_connections 2024-04-25T01:45:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.714+0000 7fed5ddf9700 1 -- 172.21.15.78:0/1651693113 wait complete. 2024-04-25T01:45:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.715+0000 7fed5ddf9700 1 Processor -- start 2024-04-25T01:45:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.715+0000 7fed5ddf9700 1 -- start start 2024-04-25T01:45:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.715+0000 7fed5ddf9700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed58100550 0x7fed58198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.715+0000 7fed5ddf9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed58103530 0x7fed58198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.715+0000 7fed5ddf9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed58103ed0 0x7fed5819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:24.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.715+0000 7fed5ddf9700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fed5810df40 con 0x7fed58100550 2024-04-25T01:45:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.715+0000 7fed5ddf9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fed5810ddc0 con 0x7fed58103ed0 2024-04-25T01:45:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.715+0000 7fed5ddf9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed5810e0c0 con 0x7fed58103530 2024-04-25T01:45:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.715+0000 7fed577fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed58100550 0x7fed58198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.715+0000 7fed577fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed58100550 0x7fed58198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42788/0 (socket says 172.21.15.78:42788) 2024-04-25T01:45:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.715+0000 7fed577fe700 1 -- 172.21.15.78:0/3326165400 learned_addr learned my addr 172.21.15.78:0/3326165400 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.716+0000 7fed56ffd700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed58103530 0x7fed58198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.716+0000 7fed57fff700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed58103ed0 0x7fed5819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.716+0000 7fed577fe700 1 -- 172.21.15.78:0/3326165400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed58103530 msgr2=0x7fed58198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.716+0000 7fed577fe700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed58103530 0x7fed58198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.716+0000 7fed577fe700 1 -- 172.21.15.78:0/3326165400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed58103ed0 msgr2=0x7fed5819fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.716+0000 7fed577fe700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed58103ed0 0x7fed5819fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.716+0000 7fed577fe700 1 -- 172.21.15.78:0/3326165400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed581a0270 con 0x7fed58100550 2024-04-25T01:45:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.716+0000 7fed57fff700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed58103ed0 0x7fed5819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:24.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.716+0000 7fed577fe700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed58100550 0x7fed58198610 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fed48000c00 tx=0x7fed48009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.716+0000 7fed54ff9700 1 -- 172.21.15.78:0/3326165400 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48027070 con 0x7fed58100550 2024-04-25T01:45:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.716+0000 7fed54ff9700 1 -- 172.21.15.78:0/3326165400 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed48004150 con 0x7fed58100550 2024-04-25T01:45:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.716+0000 7fed54ff9700 1 -- 172.21.15.78:0/3326165400 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4800f6c0 con 0x7fed58100550 2024-04-25T01:45:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.716+0000 7fed5ddf9700 1 -- 172.21.15.78:0/3326165400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed581a0500 con 0x7fed58100550 2024-04-25T01:45:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.716+0000 7fed5ddf9700 1 -- 172.21.15.78:0/3326165400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed581a0980 con 0x7fed58100550 2024-04-25T01:45:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.717+0000 7fed5ddf9700 1 -- 172.21.15.78:0/3326165400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed58068f60 con 0x7fed58100550 2024-04-25T01:45:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.718+0000 7fed54ff9700 1 -- 172.21.15.78:0/3326165400 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fed4802a070 con 0x7fed58100550 2024-04-25T01:45:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.719+0000 7fed54ff9700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fed400a2c20 0x7fed400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.719+0000 7fed56ffd700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fed400a2c20 0x7fed400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.719+0000 7fed54ff9700 1 -- 172.21.15.78:0/3326165400 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fed480d0c60 con 0x7fed58100550 2024-04-25T01:45:24.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.720+0000 7fed56ffd700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fed400a2c20 0x7fed400a50e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fed4c006fd0 tx=0x7fed4c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:24.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.723+0000 7fed54ff9700 1 -- 172.21.15.78:0/3326165400 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fed48031020 con 0x7fed58100550 2024-04-25T01:45:24.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.964+0000 7fed5ddf9700 1 -- 172.21.15.78:0/3326165400 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fed5802dd80 con 0x7fed58100550 2024-04-25T01:45:24.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.965+0000 7fed54ff9700 1 -- 172.21.15.78:0/3326165400 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3100 (secure 0 0 0) 0x7fed48013070 con 0x7fed58100550 2024-04-25T01:45:24.957 INFO:teuthology.orchestra.run.smithi078.stdout:e28 2024-04-25T01:45:24.957 INFO:teuthology.orchestra.run.smithi078.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:45:24.957 INFO:teuthology.orchestra.run.smithi078.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T01:45:24.957 INFO:teuthology.orchestra.run.smithi078.stdout:legacy client fscid: 1 2024-04-25T01:45:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:24.958 INFO:teuthology.orchestra.run.smithi078.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:45:24.958 INFO:teuthology.orchestra.run.smithi078.stdout:fs_name cephfs 2024-04-25T01:45:24.958 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 28 2024-04-25T01:45:24.958 INFO:teuthology.orchestra.run.smithi078.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T01:45:24.958 INFO:teuthology.orchestra.run.smithi078.stdout:created 2024-04-25T01:43:17.355192+0000 2024-04-25T01:45:24.958 INFO:teuthology.orchestra.run.smithi078.stdout:modified 2024-04-25T01:43:44.435289+0000 2024-04-25T01:45:24.958 INFO:teuthology.orchestra.run.smithi078.stdout:tableserver 0 2024-04-25T01:45:24.958 INFO:teuthology.orchestra.run.smithi078.stdout:root 0 2024-04-25T01:45:24.958 INFO:teuthology.orchestra.run.smithi078.stdout:session_timeout 300 2024-04-25T01:45:24.958 INFO:teuthology.orchestra.run.smithi078.stdout:session_autoclose 300 2024-04-25T01:45:24.958 INFO:teuthology.orchestra.run.smithi078.stdout:max_file_size 1099511627776 2024-04-25T01:45:24.958 INFO:teuthology.orchestra.run.smithi078.stdout:max_xattr_size 65536 2024-04-25T01:45:24.958 INFO:teuthology.orchestra.run.smithi078.stdout:required_client_features {} 2024-04-25T01:45:24.959 INFO:teuthology.orchestra.run.smithi078.stdout:last_failure 0 2024-04-25T01:45:24.959 INFO:teuthology.orchestra.run.smithi078.stdout:last_failure_osd_epoch 0 2024-04-25T01:45:24.959 INFO:teuthology.orchestra.run.smithi078.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T01:45:24.959 INFO:teuthology.orchestra.run.smithi078.stdout:max_mds 5 2024-04-25T01:45:24.959 INFO:teuthology.orchestra.run.smithi078.stdout:in 0,1,2,3,4 2024-04-25T01:45:24.959 INFO:teuthology.orchestra.run.smithi078.stdout:up {0=24491,1=24455,2=24448,3=24392,4=24379} 2024-04-25T01:45:24.959 INFO:teuthology.orchestra.run.smithi078.stdout:failed 2024-04-25T01:45:24.959 INFO:teuthology.orchestra.run.smithi078.stdout:damaged 2024-04-25T01:45:24.959 INFO:teuthology.orchestra.run.smithi078.stdout:stopped 2024-04-25T01:45:24.959 INFO:teuthology.orchestra.run.smithi078.stdout:data_pools [3,4] 2024-04-25T01:45:24.959 INFO:teuthology.orchestra.run.smithi078.stdout:metadata_pool 2 2024-04-25T01:45:24.959 INFO:teuthology.orchestra.run.smithi078.stdout:inline_data disabled 2024-04-25T01:45:24.959 INFO:teuthology.orchestra.run.smithi078.stdout:balancer 2024-04-25T01:45:24.959 INFO:teuthology.orchestra.run.smithi078.stdout:bal_rank_mask -1 2024-04-25T01:45:24.960 INFO:teuthology.orchestra.run.smithi078.stdout:standby_count_wanted 0 2024-04-25T01:45:24.960 INFO:teuthology.orchestra.run.smithi078.stdout:qdb_cluster leader: 24491 members: 24379,24491,24455,24392,24448 2024-04-25T01:45:24.960 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b{0:24491} state up:active seq 3 addr [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:45:24.960 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c{0:24427} state up:standby-replay seq 1 addr [v2:172.21.15.196:6836/1117133390,v1:172.21.15.196:6837/1117133390] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:45:24.960 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.h{1:24455} state up:active seq 12 addr [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:45:24.960 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.e{1:24419} state up:standby-replay seq 1 addr [v2:172.21.15.152:6834/4251210610,v1:172.21.15.152:6835/4251210610] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:45:24.960 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.i{2:24448} state up:active seq 11 addr [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:45:24.960 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.f{2:24409} state up:standby-replay seq 1 addr [v2:172.21.15.196:6834/2927651724,v1:172.21.15.196:6835/2927651724] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:45:24.960 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.k{3:24392} state up:active seq 16 addr [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:45:24.960 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a{3:14646} state up:standby-replay seq 1 addr [v2:172.21.15.78:6840/1963410683,v1:172.21.15.78:6841/1963410683] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:45:24.960 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.l{4:24379} state up:active seq 17 addr [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:45:24.960 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.g{4:14634} state up:standby-replay seq 1 addr [v2:172.21.15.78:6838/1337519666,v1:172.21.15.78:6839/1337519666] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:45:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:24.961 INFO:teuthology.orchestra.run.smithi078.stdout:Standby daemons: 2024-04-25T01:45:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:24.961 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.78:6834/2013480406,v1:172.21.15.78:6835/2013480406] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:45:24.961 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.78:6836/3364816743,v1:172.21.15.78:6837/3364816743] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:45:24.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.967+0000 7fed5ddf9700 1 -- 172.21.15.78:0/3326165400 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fed400a2c20 msgr2=0x7fed400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:24.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.967+0000 7fed5ddf9700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fed400a2c20 0x7fed400a50e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fed4c006fd0 tx=0x7fed4c008040 comp rx=0 tx=0).stop 2024-04-25T01:45:24.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.967+0000 7fed5ddf9700 1 -- 172.21.15.78:0/3326165400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed58100550 msgr2=0x7fed58198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:24.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.967+0000 7fed5ddf9700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed58100550 0x7fed58198610 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fed48000c00 tx=0x7fed48009ee0 comp rx=0 tx=0).stop 2024-04-25T01:45:24.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.967+0000 7fed5ddf9700 1 -- 172.21.15.78:0/3326165400 shutdown_connections 2024-04-25T01:45:24.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.967+0000 7fed5ddf9700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fed400a2c20 0x7fed400a50e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:24.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.967+0000 7fed5ddf9700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed58103ed0 0x7fed5819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:24.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.967+0000 7fed5ddf9700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed58100550 0x7fed58198610 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:24.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.967+0000 7fed5ddf9700 1 --2- 172.21.15.78:0/3326165400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed58103530 0x7fed58198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:24.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.967+0000 7fed5ddf9700 1 -- 172.21.15.78:0/3326165400 >> 172.21.15.78:0/3326165400 conn(0x7fed580fbe90 msgr2=0x7fed58106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:24.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.967+0000 7fed5ddf9700 1 -- 172.21.15.78:0/3326165400 shutdown_connections 2024-04-25T01:45:24.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:24.967+0000 7fed5ddf9700 1 -- 172.21.15.78:0/3326165400 wait complete. 2024-04-25T01:45:24.963 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-04-25T01:45:25.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:25 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3326165400' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:45:25.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:25 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3326165400' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:45:25.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:25 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3326165400' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:45:25.714 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T01:45:25.955 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:45:26.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:26 smithi152 ceph-mon[32465]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 7.8 KiB/s, 2 objects/s recovering 2024-04-25T01:45:26.357 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:26 smithi078 ceph-mon[29469]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 7.8 KiB/s, 2 objects/s recovering 2024-04-25T01:45:26.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:26 smithi196 ceph-mon[32978]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 7.8 KiB/s, 2 objects/s recovering 2024-04-25T01:45:27.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.727+0000 7f4041631700 1 -- 172.21.15.78:0/2716260051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403c102da0 msgr2=0x7f403c105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:27.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.727+0000 7f4041631700 1 --2- 172.21.15.78:0/2716260051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403c102da0 0x7f403c105190 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f402c009990 tx=0x7f402c009ca0 comp rx=0 tx=0).stop 2024-04-25T01:45:27.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.727+0000 7f4041631700 1 -- 172.21.15.78:0/2716260051 shutdown_connections 2024-04-25T01:45:27.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.727+0000 7f4041631700 1 --2- 172.21.15.78:0/2716260051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f403c108000 0x7f403c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:27.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.727+0000 7f4041631700 1 --2- 172.21.15.78:0/2716260051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403c102da0 0x7f403c105190 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.727+0000 7f4041631700 1 --2- 172.21.15.78:0/2716260051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f403c1056d0 0x7f403c107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.727+0000 7f4041631700 1 -- 172.21.15.78:0/2716260051 >> 172.21.15.78:0/2716260051 conn(0x7f403c0fc730 msgr2=0x7f403c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.727+0000 7f4041631700 1 -- 172.21.15.78:0/2716260051 shutdown_connections 2024-04-25T01:45:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.727+0000 7f4041631700 1 -- 172.21.15.78:0/2716260051 wait complete. 2024-04-25T01:45:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.728+0000 7f4041631700 1 Processor -- start 2024-04-25T01:45:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.728+0000 7f4041631700 1 -- start start 2024-04-25T01:45:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.728+0000 7f4041631700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f403c102da0 0x7f403c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.728+0000 7f4041631700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f403c1056d0 0x7f403c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.728+0000 7f4041631700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403c108000 0x7f403c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.728+0000 7f4041631700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f403c10df60 con 0x7f403c108000 2024-04-25T01:45:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.728+0000 7f4041631700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f403c10dde0 con 0x7f403c1056d0 2024-04-25T01:45:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.728+0000 7f4041631700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f403c10e0e0 con 0x7f403c102da0 2024-04-25T01:45:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.728+0000 7f403affd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f403c102da0 0x7f403c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.728+0000 7f403b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403c108000 0x7f403c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.728+0000 7f403b7fe700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403c108000 0x7f403c19fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42806/0 (socket says 172.21.15.78:42806) 2024-04-25T01:45:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.728+0000 7f403b7fe700 1 -- 172.21.15.78:0/3055606983 learned_addr learned my addr 172.21.15.78:0/3055606983 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.728+0000 7f403a7fc700 1 --2- 172.21.15.78:0/3055606983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f403c1056d0 0x7f403c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.729+0000 7f403b7fe700 1 -- 172.21.15.78:0/3055606983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f403c102da0 msgr2=0x7f403c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.729+0000 7f403b7fe700 1 --2- 172.21.15.78:0/3055606983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f403c102da0 0x7f403c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.729+0000 7f403b7fe700 1 -- 172.21.15.78:0/3055606983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f403c1056d0 msgr2=0x7f403c198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.729+0000 7f403b7fe700 1 --2- 172.21.15.78:0/3055606983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f403c1056d0 0x7f403c198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.729+0000 7f403b7fe700 1 -- 172.21.15.78:0/3055606983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c1a02c0 con 0x7f403c108000 2024-04-25T01:45:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.729+0000 7f403b7fe700 1 --2- 172.21.15.78:0/3055606983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403c108000 0x7f403c19fc20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f403400ba10 tx=0x7f403400bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.729+0000 7f4023fff700 1 -- 172.21.15.78:0/3055606983 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403400d8e0 con 0x7f403c108000 2024-04-25T01:45:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.729+0000 7f4041631700 1 -- 172.21.15.78:0/3055606983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c1a05b0 con 0x7f403c108000 2024-04-25T01:45:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.729+0000 7f4023fff700 1 -- 172.21.15.78:0/3055606983 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4034015420 con 0x7f403c108000 2024-04-25T01:45:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.729+0000 7f4023fff700 1 -- 172.21.15.78:0/3055606983 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034014630 con 0x7f403c108000 2024-04-25T01:45:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.729+0000 7f4041631700 1 -- 172.21.15.78:0/3055606983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c1a0b20 con 0x7f403c108000 2024-04-25T01:45:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.730+0000 7f4041631700 1 -- 172.21.15.78:0/3055606983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403c068f20 con 0x7f403c108000 2024-04-25T01:45:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.731+0000 7f4023fff700 1 -- 172.21.15.78:0/3055606983 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4034015980 con 0x7f403c108000 2024-04-25T01:45:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.732+0000 7f4023fff700 1 --2- 172.21.15.78:0/3055606983 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40240a2c20 0x7f40240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.732+0000 7f403affd700 1 --2- 172.21.15.78:0/3055606983 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40240a2c20 0x7f40240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.732+0000 7f4023fff700 1 -- 172.21.15.78:0/3055606983 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f40340cf160 con 0x7f403c108000 2024-04-25T01:45:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.732+0000 7f403affd700 1 --2- 172.21.15.78:0/3055606983 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40240a2c20 0x7f40240a50e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f402c009750 tx=0x7f402c003960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.735+0000 7f4023fff700 1 -- 172.21.15.78:0/3055606983 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f403409bce0 con 0x7f403c108000 2024-04-25T01:45:27.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.932+0000 7f4041631700 1 -- 172.21.15.78:0/3055606983 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f403c02db30 con 0x7f403c108000 2024-04-25T01:45:27.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.933+0000 7f4023fff700 1 -- 172.21.15.78:0/3055606983 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5719 (secure 0 0 0) 0x7f403409b5e0 con 0x7f403c108000 2024-04-25T01:45:27.925 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 94 2024-04-25T01:45:27.925 INFO:teuthology.orchestra.run.smithi078.stdout:fsid df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:45:27.925 INFO:teuthology.orchestra.run.smithi078.stdout:created 2024-04-25T01:35:22.529144+0000 2024-04-25T01:45:27.925 INFO:teuthology.orchestra.run.smithi078.stdout:modified 2024-04-25T01:45:18.989934+0000 2024-04-25T01:45:27.925 INFO:teuthology.orchestra.run.smithi078.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T01:45:27.926 INFO:teuthology.orchestra.run.smithi078.stdout:crush_version 27 2024-04-25T01:45:27.926 INFO:teuthology.orchestra.run.smithi078.stdout:full_ratio 0.95 2024-04-25T01:45:27.926 INFO:teuthology.orchestra.run.smithi078.stdout:backfillfull_ratio 0.9 2024-04-25T01:45:27.926 INFO:teuthology.orchestra.run.smithi078.stdout:nearfull_ratio 0.85 2024-04-25T01:45:27.926 INFO:teuthology.orchestra.run.smithi078.stdout:require_min_compat_client luminous 2024-04-25T01:45:27.926 INFO:teuthology.orchestra.run.smithi078.stdout:min_compat_client luminous 2024-04-25T01:45:27.926 INFO:teuthology.orchestra.run.smithi078.stdout:require_osd_release squid 2024-04-25T01:45:27.926 INFO:teuthology.orchestra.run.smithi078.stdout:stretch_mode_enabled false 2024-04-25T01:45:27.926 INFO:teuthology.orchestra.run.smithi078.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T01:45:27.926 INFO:teuthology.orchestra.run.smithi078.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-25T01:45:27.926 INFO:teuthology.orchestra.run.smithi078.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-25T01:45:27.926 INFO:teuthology.orchestra.run.smithi078.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-25T01:45:27.926 INFO:teuthology.orchestra.run.smithi078.stdout:max_osd 12 2024-04-25T01:45:27.927 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6802/519023653,v1:172.21.15.78:6803/519023653] [v2:172.21.15.78:6804/519023653,v1:172.21.15.78:6805/519023653] exists,up bab2e70b-bf93-48c8-944f-00e2d8424353 2024-04-25T01:45:27.927 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6800/2731006494,v1:172.21.15.152:6801/2731006494] [v2:172.21.15.152:6802/2731006494,v1:172.21.15.152:6803/2731006494] exists,up 54418993-9619-4854-9ce9-4bc5ff572a10 2024-04-25T01:45:27.927 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/2032225723,v1:172.21.15.196:6801/2032225723] [v2:172.21.15.196:6802/2032225723,v1:172.21.15.196:6803/2032225723] exists,up fcc6ae83-6a79-4c84-bf51-d4c6a1c15630 2024-04-25T01:45:27.927 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 up in weight 1 up_from 25 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6810/3760935265,v1:172.21.15.78:6811/3760935265] [v2:172.21.15.78:6812/3760935265,v1:172.21.15.78:6813/3760935265] exists,up 065344a4-91bb-4940-821e-1d7abd0dc292 2024-04-25T01:45:27.927 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 up in weight 1 up_from 33 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6808/2556929312,v1:172.21.15.152:6809/2556929312] [v2:172.21.15.152:6810/2556929312,v1:172.21.15.152:6811/2556929312] exists,up 6def52d6-f070-42a6-80a9-318b18cfe4a9 2024-04-25T01:45:27.927 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 up in weight 1 up_from 39 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6808/2766250253,v1:172.21.15.196:6809/2766250253] [v2:172.21.15.196:6810/2766250253,v1:172.21.15.196:6811/2766250253] exists,up 537029e6-890b-47da-973c-c3397db9751e 2024-04-25T01:45:27.927 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 up in weight 1 up_from 47 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6818/4272451125,v1:172.21.15.78:6819/4272451125] [v2:172.21.15.78:6820/4272451125,v1:172.21.15.78:6821/4272451125] exists,up 7803a823-c8f7-4015-ba43-f8c0df7eb053 2024-04-25T01:45:27.927 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 up in weight 1 up_from 52 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6816/4200178251,v1:172.21.15.152:6817/4200178251] [v2:172.21.15.152:6818/4200178251,v1:172.21.15.152:6819/4200178251] exists,up 9db92c0f-3343-4fdf-b894-f55c250c741d 2024-04-25T01:45:27.927 INFO:teuthology.orchestra.run.smithi078.stdout:osd.8 up in weight 1 up_from 60 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6816/3733356426,v1:172.21.15.196:6817/3733356426] [v2:172.21.15.196:6818/3733356426,v1:172.21.15.196:6819/3733356426] exists,up 28f5d9f1-e260-43b1-9741-2717211085e6 2024-04-25T01:45:27.927 INFO:teuthology.orchestra.run.smithi078.stdout:osd.9 up in weight 1 up_from 68 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6826/3327461277,v1:172.21.15.78:6827/3327461277] [v2:172.21.15.78:6828/3327461277,v1:172.21.15.78:6829/3327461277] exists,up 07876526-a50e-4031-ad91-552cd02a1b6e 2024-04-25T01:45:27.927 INFO:teuthology.orchestra.run.smithi078.stdout:osd.10 up in weight 1 up_from 73 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6824/3363913476,v1:172.21.15.152:6825/3363913476] [v2:172.21.15.152:6826/3363913476,v1:172.21.15.152:6827/3363913476] exists,up 9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1 2024-04-25T01:45:27.927 INFO:teuthology.orchestra.run.smithi078.stdout:osd.11 up in weight 1 up_from 81 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6824/3010681986,v1:172.21.15.196:6825/3010681986] [v2:172.21.15.196:6826/3010681986,v1:172.21.15.196:6827/3010681986] exists,up a29b5606-d549-4980-bb67-adaf71ac22bf 2024-04-25T01:45:27.927 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.6 [8,11] 2024-04-25T01:45:27.927 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.b [2,4] 2024-04-25T01:45:27.928 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.14 [2,4] 2024-04-25T01:45:27.928 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.1f [8,1,7,11] 2024-04-25T01:45:27.928 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.0 [4,8,5,9] 2024-04-25T01:45:27.928 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.1 [4,9] 2024-04-25T01:45:27.928 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.c [3,9] 2024-04-25T01:45:27.928 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.17 [4,8] 2024-04-25T01:45:27.928 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.1a [5,7] 2024-04-25T01:45:27.928 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.21 [4,6] 2024-04-25T01:45:27.928 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 4.5 [5,10] 2024-04-25T01:45:27.928 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 4.14 [9,8] 2024-04-25T01:45:27.928 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 4.16 [9,8] 2024-04-25T01:45:27.928 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 4.18 [9,8] 2024-04-25T01:45:27.928 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6801/1101495738 expires 2024-04-26T01:36:16.812502+0000 2024-04-25T01:45:27.929 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6800/1101495738 expires 2024-04-26T01:36:16.812502+0000 2024-04-25T01:45:27.929 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/2762765405 expires 2024-04-26T01:36:16.812502+0000 2024-04-25T01:45:27.929 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/4170716207 expires 2024-04-26T01:35:50.507447+0000 2024-04-25T01:45:27.929 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/1136116795 expires 2024-04-26T01:35:50.507447+0000 2024-04-25T01:45:27.929 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6801/1544860340 expires 2024-04-26T01:35:50.507447+0000 2024-04-25T01:45:27.929 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6800/1544860340 expires 2024-04-26T01:35:50.507447+0000 2024-04-25T01:45:27.929 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/211357528 expires 2024-04-26T01:36:16.812502+0000 2024-04-25T01:45:27.929 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/88994848 expires 2024-04-26T01:36:16.812502+0000 2024-04-25T01:45:27.929 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/1196975031 expires 2024-04-26T01:35:50.507447+0000 2024-04-25T01:45:27.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.935+0000 7f4041631700 1 -- 172.21.15.78:0/3055606983 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40240a2c20 msgr2=0x7f40240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:27.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.935+0000 7f4041631700 1 --2- 172.21.15.78:0/3055606983 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40240a2c20 0x7f40240a50e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f402c009750 tx=0x7f402c003960 comp rx=0 tx=0).stop 2024-04-25T01:45:27.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.935+0000 7f4041631700 1 -- 172.21.15.78:0/3055606983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403c108000 msgr2=0x7f403c19fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:27.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.935+0000 7f4041631700 1 --2- 172.21.15.78:0/3055606983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403c108000 0x7f403c19fc20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f403400ba10 tx=0x7f403400bd20 comp rx=0 tx=0).stop 2024-04-25T01:45:27.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.935+0000 7f4041631700 1 -- 172.21.15.78:0/3055606983 shutdown_connections 2024-04-25T01:45:27.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.935+0000 7f4041631700 1 --2- 172.21.15.78:0/3055606983 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f40240a2c20 0x7f40240a50e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:27.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.935+0000 7f4041631700 1 --2- 172.21.15.78:0/3055606983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f403c102da0 0x7f403c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:27.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.935+0000 7f4041631700 1 --2- 172.21.15.78:0/3055606983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f403c108000 0x7f403c19fc20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:27.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.935+0000 7f4041631700 1 --2- 172.21.15.78:0/3055606983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f403c1056d0 0x7f403c198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:27.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.935+0000 7f4041631700 1 -- 172.21.15.78:0/3055606983 >> 172.21.15.78:0/3055606983 conn(0x7f403c0fc730 msgr2=0x7f403c1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:27.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.935+0000 7f4041631700 1 -- 172.21.15.78:0/3055606983 shutdown_connections 2024-04-25T01:45:27.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:27.935+0000 7f4041631700 1 -- 172.21.15.78:0/3055606983 wait complete. 2024-04-25T01:45:28.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:28 smithi152 ceph-mon[32465]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 7.0 KiB/s, 2 objects/s recovering 2024-04-25T01:45:28.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:28 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3055606983' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T01:45:28.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:28 smithi196 ceph-mon[32978]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 7.0 KiB/s, 2 objects/s recovering 2024-04-25T01:45:28.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:28 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3055606983' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T01:45:28.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:28 smithi078 ceph-mon[29469]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 7.0 KiB/s, 2 objects/s recovering 2024-04-25T01:45:28.388 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:28 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3055606983' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T01:45:28.710 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T01:45:28.723 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:45:28.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.833+0000 7f228eaee640 1 -- 172.21.15.78:0/4162726379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2288100680 msgr2=0x7f2288100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:28.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.833+0000 7f228eaee640 1 --2- 172.21.15.78:0/4162726379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2288100680 0x7f2288100aa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2278009900 tx=0x7f227802a410 comp rx=0 tx=0).stop 2024-04-25T01:45:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.833+0000 7f228eaee640 1 -- 172.21.15.78:0/4162726379 shutdown_connections 2024-04-25T01:45:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.833+0000 7f228eaee640 1 --2- 172.21.15.78:0/4162726379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2288102230 0x7f2288108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.833+0000 7f228eaee640 1 --2- 172.21.15.78:0/4162726379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2288101870 0x7f2288101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.833+0000 7f228eaee640 1 --2- 172.21.15.78:0/4162726379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2288100680 0x7f2288100aa0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.833+0000 7f228eaee640 1 -- 172.21.15.78:0/4162726379 >> 172.21.15.78:0/4162726379 conn(0x7f22880fbbf0 msgr2=0x7f22880fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.833+0000 7f228eaee640 1 -- 172.21.15.78:0/4162726379 shutdown_connections 2024-04-25T01:45:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.833+0000 7f228eaee640 1 -- 172.21.15.78:0/4162726379 wait complete. 2024-04-25T01:45:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.833+0000 7f228eaee640 1 Processor -- start 2024-04-25T01:45:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.833+0000 7f228eaee640 1 -- start start 2024-04-25T01:45:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228eaee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2288100680 0x7f2288069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228eaee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2288101870 0x7f228819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228eaee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2288102230 0x7f22881a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228eaee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f228810c300 con 0x7f2288102230 2024-04-25T01:45:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228eaee640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f228810c180 con 0x7f2288100680 2024-04-25T01:45:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228eaee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f228810c480 con 0x7f2288101870 2024-04-25T01:45:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228d08b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2288102230 0x7f22881a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228d08b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2288102230 0x7f22881a1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42820/0 (socket says 172.21.15.78:42820) 2024-04-25T01:45:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228d08b640 1 -- 172.21.15.78:0/1030747412 learned_addr learned my addr 172.21.15.78:0/1030747412 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f2287fff640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2288101870 0x7f228819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228c88a640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2288100680 0x7f2288069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228d08b640 1 -- 172.21.15.78:0/1030747412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2288101870 msgr2=0x7f228819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228d08b640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2288101870 0x7f228819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228d08b640 1 -- 172.21.15.78:0/1030747412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2288100680 msgr2=0x7f2288069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228d08b640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2288100680 0x7f2288069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228d08b640 1 -- 172.21.15.78:0/1030747412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22881a1900 con 0x7f2288102230 2024-04-25T01:45:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228c88a640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2288100680 0x7f2288069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228d08b640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2288102230 0x7f22881a1110 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f228000eaa0 tx=0x7f228000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:28.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f2285ffb640 1 -- 172.21.15.78:0/1030747412 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228000cdb0 con 0x7f2288102230 2024-04-25T01:45:28.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f2285ffb640 1 -- 172.21.15.78:0/1030747412 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2280005020 con 0x7f2288102230 2024-04-25T01:45:28.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f228eaee640 1 -- 172.21.15.78:0/1030747412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22881a1b90 con 0x7f2288102230 2024-04-25T01:45:28.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.834+0000 7f2285ffb640 1 -- 172.21.15.78:0/1030747412 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22800116b0 con 0x7f2288102230 2024-04-25T01:45:28.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.835+0000 7f228eaee640 1 -- 172.21.15.78:0/1030747412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22881a2130 con 0x7f2288102230 2024-04-25T01:45:28.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.836+0000 7f228eaee640 1 -- 172.21.15.78:0/1030747412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22881079a0 con 0x7f2288102230 2024-04-25T01:45:28.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.836+0000 7f2285ffb640 1 -- 172.21.15.78:0/1030747412 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2280005560 con 0x7f2288102230 2024-04-25T01:45:28.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.837+0000 7f2285ffb640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f22700a2c40 0x7f22700a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:28.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.837+0000 7f228c88a640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f22700a2c40 0x7f22700a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:28.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.837+0000 7f2285ffb640 1 -- 172.21.15.78:0/1030747412 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f22800cf580 con 0x7f2288102230 2024-04-25T01:45:28.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.838+0000 7f228c88a640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f22700a2c40 0x7f22700a5100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2278006010 tx=0x7f2278032cf0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:28.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:28.842+0000 7f2285ffb640 1 -- 172.21.15.78:0/1030747412 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2280016030 con 0x7f2288102230 2024-04-25T01:45:29.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.020+0000 7f228eaee640 1 -- 172.21.15.78:0/1030747412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2288062340 con 0x7f2288102230 2024-04-25T01:45:29.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.022+0000 7f2285ffb640 1 -- 172.21.15.78:0/1030747412 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24844 (secure 0 0 0) 0x7f228009c100 con 0x7f2288102230 2024-04-25T01:45:29.015 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:29.015 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":94,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:45:18.989934+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:45:29.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.024+0000 7f228eaee640 1 -- 172.21.15.78:0/1030747412 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f22700a2c40 msgr2=0x7f22700a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.024+0000 7f228eaee640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f22700a2c40 0x7f22700a5100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2278006010 tx=0x7f2278032cf0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.024+0000 7f228eaee640 1 -- 172.21.15.78:0/1030747412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2288102230 msgr2=0x7f22881a1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.024+0000 7f228eaee640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2288102230 0x7f22881a1110 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f228000eaa0 tx=0x7f228000ef60 comp rx=0 tx=0).stop 2024-04-25T01:45:29.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.024+0000 7f228eaee640 1 -- 172.21.15.78:0/1030747412 shutdown_connections 2024-04-25T01:45:29.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.024+0000 7f228eaee640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f22700a2c40 0x7f22700a5100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.024+0000 7f228eaee640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2288102230 0x7f22881a1110 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.024+0000 7f228eaee640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2288101870 0x7f228819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.024+0000 7f228eaee640 1 --2- 172.21.15.78:0/1030747412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2288100680 0x7f2288069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.024+0000 7f228eaee640 1 -- 172.21.15.78:0/1030747412 >> 172.21.15.78:0/1030747412 conn(0x7f22880fbbf0 msgr2=0x7f2288104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:29.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.024+0000 7f228eaee640 1 -- 172.21.15.78:0/1030747412 shutdown_connections 2024-04-25T01:45:29.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.024+0000 7f228eaee640 1 -- 172.21.15.78:0/1030747412 wait complete. 2024-04-25T01:45:29.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:38:39.136418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:43:15.613881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:43:16.600270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:43:19.650457+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:45:29.029 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:45:29.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.137+0000 7fd7d7f22640 1 -- 172.21.15.78:0/3697022001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d0101870 msgr2=0x7fd7d0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.137+0000 7fd7d7f22640 1 --2- 172.21.15.78:0/3697022001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d0101870 0x7fd7d0101cf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009900 tx=0x7fd7c002a410 comp rx=0 tx=0).stop 2024-04-25T01:45:29.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.137+0000 7fd7d7f22640 1 -- 172.21.15.78:0/3697022001 shutdown_connections 2024-04-25T01:45:29.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.137+0000 7fd7d7f22640 1 --2- 172.21.15.78:0/3697022001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d0102230 0x7fd7d0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.137+0000 7fd7d7f22640 1 --2- 172.21.15.78:0/3697022001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d0101870 0x7fd7d0101cf0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.137+0000 7fd7d7f22640 1 --2- 172.21.15.78:0/3697022001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7d0100680 0x7fd7d0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.137+0000 7fd7d7f22640 1 -- 172.21.15.78:0/3697022001 >> 172.21.15.78:0/3697022001 conn(0x7fd7d00fbbf0 msgr2=0x7fd7d00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:29.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.137+0000 7fd7d7f22640 1 -- 172.21.15.78:0/3697022001 shutdown_connections 2024-04-25T01:45:29.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.137+0000 7fd7d7f22640 1 -- 172.21.15.78:0/3697022001 wait complete. 2024-04-25T01:45:29.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.137+0000 7fd7d7f22640 1 Processor -- start 2024-04-25T01:45:29.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.137+0000 7fd7d7f22640 1 -- start start 2024-04-25T01:45:29.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d7f22640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d0100680 0x7fd7d0197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:29.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d7f22640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d0101870 0x7fd7d0197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:29.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d7f22640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7d0102230 0x7fd7d019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:29.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d7f22640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d010d450 con 0x7fd7d0101870 2024-04-25T01:45:29.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d7f22640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d010d2d0 con 0x7fd7d0102230 2024-04-25T01:45:29.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d7f22640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d010d5d0 con 0x7fd7d0100680 2024-04-25T01:45:29.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d54bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d0101870 0x7fd7d0197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:29.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d54bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d0101870 0x7fd7d0197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42826/0 (socket says 172.21.15.78:42826) 2024-04-25T01:45:29.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d54bd640 1 -- 172.21.15.78:0/1967251603 learned_addr learned my addr 172.21.15.78:0/1967251603 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:29.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d5cbe640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d0100680 0x7fd7d0197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:29.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d64bf640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7d0102230 0x7fd7d019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:29.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d54bd640 1 -- 172.21.15.78:0/1967251603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d0100680 msgr2=0x7fd7d0197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d54bd640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d0100680 0x7fd7d0197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d54bd640 1 -- 172.21.15.78:0/1967251603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7d0102230 msgr2=0x7fd7d019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d54bd640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7d0102230 0x7fd7d019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d54bd640 1 -- 172.21.15.78:0/1967251603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d019f6c0 con 0x7fd7d0101870 2024-04-25T01:45:29.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d5cbe640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d0100680 0x7fd7d0197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:29.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d64bf640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7d0102230 0x7fd7d019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:29.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d54bd640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d0101870 0x7fd7d0197fe0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009230 tx=0x7fd7c0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:29.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7c6ffd640 1 -- 172.21.15.78:0/1967251603 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c003f030 con 0x7fd7d0101870 2024-04-25T01:45:29.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7c6ffd640 1 -- 172.21.15.78:0/1967251603 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7c0003ce0 con 0x7fd7d0101870 2024-04-25T01:45:29.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d7f22640 1 -- 172.21.15.78:0/1967251603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d019f950 con 0x7fd7d0101870 2024-04-25T01:45:29.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7c6ffd640 1 -- 172.21.15.78:0/1967251603 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0035070 con 0x7fd7d0101870 2024-04-25T01:45:29.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.138+0000 7fd7d7f22640 1 -- 172.21.15.78:0/1967251603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d019fec0 con 0x7fd7d0101870 2024-04-25T01:45:29.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.139+0000 7fd7d7f22640 1 -- 172.21.15.78:0/1967251603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d01079a0 con 0x7fd7d0101870 2024-04-25T01:45:29.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.140+0000 7fd7c6ffd640 1 -- 172.21.15.78:0/1967251603 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd7c0042030 con 0x7fd7d0101870 2024-04-25T01:45:29.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.141+0000 7fd7c6ffd640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd7bc0a2b90 0x7fd7bc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:29.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.141+0000 7fd7d5cbe640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd7bc0a2b90 0x7fd7bc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:29.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.141+0000 7fd7c6ffd640 1 -- 172.21.15.78:0/1967251603 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fd7c00efc40 con 0x7fd7d0101870 2024-04-25T01:45:29.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.142+0000 7fd7d5cbe640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd7bc0a2b90 0x7fd7bc0a5050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd7d01016d0 tx=0x7fd7c8008370 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:29.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.146+0000 7fd7c6ffd640 1 -- 172.21.15.78:0/1967251603 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd7c00bc7c0 con 0x7fd7d0101870 2024-04-25T01:45:29.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:29 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1030747412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:29.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.324+0000 7fd7d7f22640 1 -- 172.21.15.78:0/1967251603 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd7d00623e0 con 0x7fd7d0101870 2024-04-25T01:45:29.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.324+0000 7fd7c6ffd640 1 -- 172.21.15.78:0/1967251603 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fd7c00bc0c0 con 0x7fd7d0101870 2024-04-25T01:45:29.317 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T01:45:29.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.326+0000 7fd7d7f22640 1 -- 172.21.15.78:0/1967251603 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd7bc0a2b90 msgr2=0x7fd7bc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.326+0000 7fd7d7f22640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd7bc0a2b90 0x7fd7bc0a5050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd7d01016d0 tx=0x7fd7c8008370 comp rx=0 tx=0).stop 2024-04-25T01:45:29.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.326+0000 7fd7d7f22640 1 -- 172.21.15.78:0/1967251603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d0101870 msgr2=0x7fd7d0197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.326+0000 7fd7d7f22640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d0101870 0x7fd7d0197fe0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009230 tx=0x7fd7c0004430 comp rx=0 tx=0).stop 2024-04-25T01:45:29.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.326+0000 7fd7d7f22640 1 -- 172.21.15.78:0/1967251603 shutdown_connections 2024-04-25T01:45:29.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.326+0000 7fd7d7f22640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7d0102230 0x7fd7d019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.326+0000 7fd7d7f22640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd7bc0a2b90 0x7fd7bc0a5050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.326+0000 7fd7d7f22640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d0101870 0x7fd7d0197fe0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.326+0000 7fd7d7f22640 1 --2- 172.21.15.78:0/1967251603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7d0100680 0x7fd7d0197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.326+0000 7fd7d7f22640 1 -- 172.21.15.78:0/1967251603 >> 172.21.15.78:0/1967251603 conn(0x7fd7d00fbbf0 msgr2=0x7fd7d0104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:29.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.326+0000 7fd7d7f22640 1 -- 172.21.15.78:0/1967251603 shutdown_connections 2024-04-25T01:45:29.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.326+0000 7fd7d7f22640 1 -- 172.21.15.78:0/1967251603 wait complete. 2024-04-25T01:45:29.329 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:29 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1030747412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:29.330 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:45:29.367 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:29 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1030747412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:29.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.437+0000 7fe75d2b2640 1 -- 172.21.15.78:0/2801707641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe758107460 msgr2=0x7fe7581098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.437+0000 7fe75d2b2640 1 --2- 172.21.15.78:0/2801707641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe758107460 0x7fe7581098b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe750009900 tx=0x7fe75002a410 comp rx=0 tx=0).stop 2024-04-25T01:45:29.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.437+0000 7fe75d2b2640 1 -- 172.21.15.78:0/2801707641 shutdown_connections 2024-04-25T01:45:29.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.437+0000 7fe75d2b2640 1 --2- 172.21.15.78:0/2801707641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe758107460 0x7fe7581098b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.437+0000 7fe75d2b2640 1 --2- 172.21.15.78:0/2801707641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe758104b30 0x7fe758106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.437+0000 7fe75d2b2640 1 --2- 172.21.15.78:0/2801707641 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe758102200 0x7fe7581045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.437+0000 7fe75d2b2640 1 -- 172.21.15.78:0/2801707641 >> 172.21.15.78:0/2801707641 conn(0x7fe7580fbbf0 msgr2=0x7fe7580fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:29.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.437+0000 7fe75d2b2640 1 -- 172.21.15.78:0/2801707641 shutdown_connections 2024-04-25T01:45:29.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.437+0000 7fe75d2b2640 1 -- 172.21.15.78:0/2801707641 wait complete. 2024-04-25T01:45:29.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.438+0000 7fe75d2b2640 1 Processor -- start 2024-04-25T01:45:29.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.438+0000 7fe75d2b2640 1 -- start start 2024-04-25T01:45:29.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.438+0000 7fe75d2b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe758102200 0x7fe758197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:29.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.438+0000 7fe75d2b2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe758104b30 0x7fe758197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:29.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.438+0000 7fe75d2b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe758107460 0x7fe75819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:29.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.438+0000 7fe75d2b2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe75810d420 con 0x7fe758107460 2024-04-25T01:45:29.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.438+0000 7fe75d2b2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe75810d2a0 con 0x7fe758104b30 2024-04-25T01:45:29.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.438+0000 7fe75d2b2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe75810d5a0 con 0x7fe758102200 2024-04-25T01:45:29.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.438+0000 7fe7577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe758107460 0x7fe75819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:29.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.438+0000 7fe7577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe758107460 0x7fe75819f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42852/0 (socket says 172.21.15.78:42852) 2024-04-25T01:45:29.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.438+0000 7fe7577fe640 1 -- 172.21.15.78:0/993067183 learned_addr learned my addr 172.21.15.78:0/993067183 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:29.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.438+0000 7fe7567fc640 1 --2- 172.21.15.78:0/993067183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe758104b30 0x7fe758197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:29.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.438+0000 7fe756ffd640 1 --2- 172.21.15.78:0/993067183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe758102200 0x7fe758197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:29.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.439+0000 7fe7577fe640 1 -- 172.21.15.78:0/993067183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe758102200 msgr2=0x7fe758197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.439+0000 7fe7577fe640 1 --2- 172.21.15.78:0/993067183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe758102200 0x7fe758197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.439+0000 7fe7577fe640 1 -- 172.21.15.78:0/993067183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe758104b30 msgr2=0x7fe758197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.439+0000 7fe7577fe640 1 --2- 172.21.15.78:0/993067183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe758104b30 0x7fe758197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.439+0000 7fe7577fe640 1 -- 172.21.15.78:0/993067183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75819f6b0 con 0x7fe758107460 2024-04-25T01:45:29.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.439+0000 7fe7577fe640 1 --2- 172.21.15.78:0/993067183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe758107460 0x7fe75819f040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe7500314e0 tx=0x7fe750004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:29.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.439+0000 7fe73ffff640 1 -- 172.21.15.78:0/993067183 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75003f030 con 0x7fe758107460 2024-04-25T01:45:29.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.439+0000 7fe73ffff640 1 -- 172.21.15.78:0/993067183 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe750003ce0 con 0x7fe758107460 2024-04-25T01:45:29.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.439+0000 7fe75d2b2640 1 -- 172.21.15.78:0/993067183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75819f940 con 0x7fe758107460 2024-04-25T01:45:29.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.439+0000 7fe73ffff640 1 -- 172.21.15.78:0/993067183 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750035070 con 0x7fe758107460 2024-04-25T01:45:29.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.439+0000 7fe75d2b2640 1 -- 172.21.15.78:0/993067183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75819fe80 con 0x7fe758107460 2024-04-25T01:45:29.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.440+0000 7fe75d2b2640 1 -- 172.21.15.78:0/993067183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7580fd6e0 con 0x7fe758107460 2024-04-25T01:45:29.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.441+0000 7fe73ffff640 1 -- 172.21.15.78:0/993067183 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe750042030 con 0x7fe758107460 2024-04-25T01:45:29.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.442+0000 7fe73ffff640 1 --2- 172.21.15.78:0/993067183 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe7400a2c10 0x7fe7400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:29.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.442+0000 7fe756ffd640 1 --2- 172.21.15.78:0/993067183 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe7400a2c10 0x7fe7400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:29.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.442+0000 7fe73ffff640 1 -- 172.21.15.78:0/993067183 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fe7500efd40 con 0x7fe758107460 2024-04-25T01:45:29.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.442+0000 7fe756ffd640 1 --2- 172.21.15.78:0/993067183 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe7400a2c10 0x7fe7400a50d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe748005c40 tx=0x7fe74800a390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:29.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.446+0000 7fe73ffff640 1 -- 172.21.15.78:0/993067183 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe7500bc8c0 con 0x7fe758107460 2024-04-25T01:45:29.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.625+0000 7fe75d2b2640 1 -- 172.21.15.78:0/993067183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe75802d170 con 0x7fe758107460 2024-04-25T01:45:29.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.626+0000 7fe73ffff640 1 -- 172.21.15.78:0/993067183 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fe7500bc1c0 con 0x7fe758107460 2024-04-25T01:45:29.618 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:45:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.628+0000 7fe75d2b2640 1 -- 172.21.15.78:0/993067183 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe7400a2c10 msgr2=0x7fe7400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.628+0000 7fe75d2b2640 1 --2- 172.21.15.78:0/993067183 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe7400a2c10 0x7fe7400a50d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe748005c40 tx=0x7fe74800a390 comp rx=0 tx=0).stop 2024-04-25T01:45:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.628+0000 7fe75d2b2640 1 -- 172.21.15.78:0/993067183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe758107460 msgr2=0x7fe75819f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.628+0000 7fe75d2b2640 1 --2- 172.21.15.78:0/993067183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe758107460 0x7fe75819f040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe7500314e0 tx=0x7fe750004360 comp rx=0 tx=0).stop 2024-04-25T01:45:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.628+0000 7fe75d2b2640 1 -- 172.21.15.78:0/993067183 shutdown_connections 2024-04-25T01:45:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.628+0000 7fe75d2b2640 1 --2- 172.21.15.78:0/993067183 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe7400a2c10 0x7fe7400a50d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.628+0000 7fe75d2b2640 1 --2- 172.21.15.78:0/993067183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe758107460 0x7fe75819f040 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.628+0000 7fe75d2b2640 1 --2- 172.21.15.78:0/993067183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe758104b30 0x7fe758197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.628+0000 7fe75d2b2640 1 --2- 172.21.15.78:0/993067183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe758102200 0x7fe758197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.628+0000 7fe75d2b2640 1 -- 172.21.15.78:0/993067183 >> 172.21.15.78:0/993067183 conn(0x7fe7580fbbf0 msgr2=0x7fe758102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.628+0000 7fe75d2b2640 1 -- 172.21.15.78:0/993067183 shutdown_connections 2024-04-25T01:45:29.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.628+0000 7fe75d2b2640 1 -- 172.21.15.78:0/993067183 wait complete. 2024-04-25T01:45:29.632 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:45:29.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/1670539636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b08102230 msgr2=0x7f5b08108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 --2- 172.21.15.78:0/1670539636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b08102230 0x7f5b08108ac0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009ca0 tx=0x7f5b0002a8c0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/1670539636 shutdown_connections 2024-04-25T01:45:29.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 --2- 172.21.15.78:0/1670539636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b08102230 0x7f5b08108ac0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 --2- 172.21.15.78:0/1670539636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b08101870 0x7f5b08101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 --2- 172.21.15.78:0/1670539636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b08100680 0x7f5b08100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/1670539636 >> 172.21.15.78:0/1670539636 conn(0x7f5b080fbbf0 msgr2=0x7f5b080fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:29.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/1670539636 shutdown_connections 2024-04-25T01:45:29.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/1670539636 wait complete. 2024-04-25T01:45:29.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 Processor -- start 2024-04-25T01:45:29.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 -- start start 2024-04-25T01:45:29.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b08100680 0x7f5b08069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:29.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b08101870 0x7f5b0819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:29.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b08102230 0x7f5b0819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:29.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0810c2e0 con 0x7f5b08102230 2024-04-25T01:45:29.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0810c160 con 0x7f5b08100680 2024-04-25T01:45:29.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0fe2c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0810c460 con 0x7f5b08101870 2024-04-25T01:45:29.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0e3c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b08102230 0x7f5b0819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:29.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0e3c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b08102230 0x7f5b0819edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59990/0 (socket says 172.21.15.78:59990) 2024-04-25T01:45:29.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.741+0000 7f5b0e3c9640 1 -- 172.21.15.78:0/3007571109 learned_addr learned my addr 172.21.15.78:0/3007571109 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:29.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.742+0000 7f5b0dbc8640 1 --2- 172.21.15.78:0/3007571109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b08100680 0x7f5b08069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:29.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.742+0000 7f5b0e3c9640 1 -- 172.21.15.78:0/3007571109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b08101870 msgr2=0x7f5b0819c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.742+0000 7f5b0e3c9640 1 --2- 172.21.15.78:0/3007571109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b08101870 0x7f5b0819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.742+0000 7f5b0e3c9640 1 -- 172.21.15.78:0/3007571109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b08100680 msgr2=0x7f5b08069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.742+0000 7f5b0e3c9640 1 --2- 172.21.15.78:0/3007571109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b08100680 0x7f5b08069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.742+0000 7f5b0e3c9640 1 -- 172.21.15.78:0/3007571109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0819f670 con 0x7f5b08102230 2024-04-25T01:45:29.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.742+0000 7f5b0dbc8640 1 --2- 172.21.15.78:0/3007571109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b08100680 0x7f5b08069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:29.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.742+0000 7f5b0e3c9640 1 --2- 172.21.15.78:0/3007571109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b08102230 0x7f5b0819edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009430 tx=0x7f5b0002f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:29.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.742+0000 7f5afeffd640 1 -- 172.21.15.78:0/3007571109 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00003c30 con 0x7f5b08102230 2024-04-25T01:45:29.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.742+0000 7f5afeffd640 1 -- 172.21.15.78:0/3007571109 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b00003dd0 con 0x7f5b08102230 2024-04-25T01:45:29.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.742+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/3007571109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0819f900 con 0x7f5b08102230 2024-04-25T01:45:29.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.742+0000 7f5afeffd640 1 -- 172.21.15.78:0/3007571109 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b000047e0 con 0x7f5b08102230 2024-04-25T01:45:29.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.742+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/3007571109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b0819fea0 con 0x7f5b08102230 2024-04-25T01:45:29.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.743+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/3007571109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b08107940 con 0x7f5b08102230 2024-04-25T01:45:29.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.744+0000 7f5afeffd640 1 -- 172.21.15.78:0/3007571109 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5b0002f800 con 0x7f5b08102230 2024-04-25T01:45:29.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.744+0000 7f5afeffd640 1 --2- 172.21.15.78:0/3007571109 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5af40a2c10 0x7f5af40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:29.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.745+0000 7f5b0dbc8640 1 --2- 172.21.15.78:0/3007571109 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5af40a2c10 0x7f5af40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:29.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.745+0000 7f5afeffd640 1 -- 172.21.15.78:0/3007571109 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f5b000f08f0 con 0x7f5b08102230 2024-04-25T01:45:29.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.745+0000 7f5b0dbc8640 1 --2- 172.21.15.78:0/3007571109 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5af40a2c10 0x7f5af40a50d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5b081016d0 tx=0x7f5b04005db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:29.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.749+0000 7f5afeffd640 1 -- 172.21.15.78:0/3007571109 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5b000bd470 con 0x7f5b08102230 2024-04-25T01:45:29.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.926+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/3007571109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5b080623e0 con 0x7f5b08102230 2024-04-25T01:45:29.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.926+0000 7f5afeffd640 1 -- 172.21.15.78:0/3007571109 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f5b000bcd70 con 0x7f5b08102230 2024-04-25T01:45:29.918 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T01:45:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.928+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/3007571109 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5af40a2c10 msgr2=0x7f5af40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.928+0000 7f5b0fe2c640 1 --2- 172.21.15.78:0/3007571109 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5af40a2c10 0x7f5af40a50d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5b081016d0 tx=0x7f5b04005db0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.928+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/3007571109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b08102230 msgr2=0x7f5b0819edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.928+0000 7f5b0fe2c640 1 --2- 172.21.15.78:0/3007571109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b08102230 0x7f5b0819edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009430 tx=0x7f5b0002f610 comp rx=0 tx=0).stop 2024-04-25T01:45:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.928+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/3007571109 shutdown_connections 2024-04-25T01:45:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.928+0000 7f5b0fe2c640 1 --2- 172.21.15.78:0/3007571109 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5af40a2c10 0x7f5af40a50d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.928+0000 7f5b0fe2c640 1 --2- 172.21.15.78:0/3007571109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b08102230 0x7f5b0819edb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.928+0000 7f5b0fe2c640 1 --2- 172.21.15.78:0/3007571109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b08101870 0x7f5b0819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.928+0000 7f5b0fe2c640 1 --2- 172.21.15.78:0/3007571109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b08100680 0x7f5b08069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.928+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/3007571109 >> 172.21.15.78:0/3007571109 conn(0x7f5b080fbbf0 msgr2=0x7f5b08104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.928+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/3007571109 shutdown_connections 2024-04-25T01:45:29.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:29.928+0000 7f5b0fe2c640 1 -- 172.21.15.78:0/3007571109 wait complete. 2024-04-25T01:45:29.931 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:45:30.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.039+0000 7f77fbd63640 1 -- 172.21.15.78:0/3913856896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77f41006a0 msgr2=0x7f77f4100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.039+0000 7f77fbd63640 1 --2- 172.21.15.78:0/3913856896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77f41006a0 0x7f77f4100ac0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009900 tx=0x7f77f002a410 comp rx=0 tx=0).stop 2024-04-25T01:45:30.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.039+0000 7f77fbd63640 1 -- 172.21.15.78:0/3913856896 shutdown_connections 2024-04-25T01:45:30.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.039+0000 7f77fbd63640 1 --2- 172.21.15.78:0/3913856896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77f4102250 0x7f77f4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.039+0000 7f77fbd63640 1 --2- 172.21.15.78:0/3913856896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f4101890 0x7f77f4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.039+0000 7f77fbd63640 1 --2- 172.21.15.78:0/3913856896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77f41006a0 0x7f77f4100ac0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.039+0000 7f77fbd63640 1 -- 172.21.15.78:0/3913856896 >> 172.21.15.78:0/3913856896 conn(0x7f77f40fbc30 msgr2=0x7f77f40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:30.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.040+0000 7f77fbd63640 1 -- 172.21.15.78:0/3913856896 shutdown_connections 2024-04-25T01:45:30.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.040+0000 7f77fbd63640 1 -- 172.21.15.78:0/3913856896 wait complete. 2024-04-25T01:45:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.040+0000 7f77fbd63640 1 Processor -- start 2024-04-25T01:45:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.040+0000 7f77fbd63640 1 -- start start 2024-04-25T01:45:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.040+0000 7f77fbd63640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77f41006a0 0x7f77f41969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.040+0000 7f77fbd63640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f4101890 0x7f77f4196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.040+0000 7f77fbd63640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77f4102250 0x7f77f419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.040+0000 7f77fbd63640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77f410c340 con 0x7f77f41006a0 2024-04-25T01:45:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.040+0000 7f77fbd63640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f77f410c1c0 con 0x7f77f4101890 2024-04-25T01:45:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.040+0000 7f77fbd63640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77f410c4c0 con 0x7f77f4102250 2024-04-25T01:45:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.040+0000 7f77f9aff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77f41006a0 0x7f77f41969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.040+0000 7f77f9aff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77f41006a0 0x7f77f41969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60014/0 (socket says 172.21.15.78:60014) 2024-04-25T01:45:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.040+0000 7f77f9aff640 1 -- 172.21.15.78:0/2961631294 learned_addr learned my addr 172.21.15.78:0/2961631294 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.041+0000 7f77f92fe640 1 --2- 172.21.15.78:0/2961631294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f4101890 0x7f77f4196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.041+0000 7f77f9aff640 1 -- 172.21.15.78:0/2961631294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77f4102250 msgr2=0x7f77f419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.041+0000 7f77f9aff640 1 --2- 172.21.15.78:0/2961631294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77f4102250 0x7f77f419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.041+0000 7f77f9aff640 1 -- 172.21.15.78:0/2961631294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f4101890 msgr2=0x7f77f4196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.041+0000 7f77f9aff640 1 --2- 172.21.15.78:0/2961631294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f4101890 0x7f77f4196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.041+0000 7f77f9aff640 1 -- 172.21.15.78:0/2961631294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f419e600 con 0x7f77f41006a0 2024-04-25T01:45:30.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.041+0000 7f77f92fe640 1 --2- 172.21.15.78:0/2961631294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f4101890 0x7f77f4196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:30.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.041+0000 7f77f9aff640 1 --2- 172.21.15.78:0/2961631294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77f41006a0 0x7f77f41969e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f77f0031440 tx=0x7f77f0032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:30.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.041+0000 7f77eaffd640 1 -- 172.21.15.78:0/2961631294 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f003f030 con 0x7f77f41006a0 2024-04-25T01:45:30.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.041+0000 7f77eaffd640 1 -- 172.21.15.78:0/2961631294 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f0003ce0 con 0x7f77f41006a0 2024-04-25T01:45:30.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.041+0000 7f77fbd63640 1 -- 172.21.15.78:0/2961631294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f419e890 con 0x7f77f41006a0 2024-04-25T01:45:30.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.041+0000 7f77eaffd640 1 -- 172.21.15.78:0/2961631294 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0035070 con 0x7f77f41006a0 2024-04-25T01:45:30.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.041+0000 7f77fbd63640 1 -- 172.21.15.78:0/2961631294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f419ed80 con 0x7f77f41006a0 2024-04-25T01:45:30.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.042+0000 7f77fbd63640 1 -- 172.21.15.78:0/2961631294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f4107a90 con 0x7f77f41006a0 2024-04-25T01:45:30.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.043+0000 7f77eaffd640 1 -- 172.21.15.78:0/2961631294 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f77f0042030 con 0x7f77f41006a0 2024-04-25T01:45:30.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.044+0000 7f77eaffd640 1 --2- 172.21.15.78:0/2961631294 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f77e00a2b90 0x7f77e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.044+0000 7f77f92fe640 1 --2- 172.21.15.78:0/2961631294 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f77e00a2b90 0x7f77e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.044+0000 7f77eaffd640 1 -- 172.21.15.78:0/2961631294 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f77f00eff50 con 0x7f77f41006a0 2024-04-25T01:45:30.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.044+0000 7f77f92fe640 1 --2- 172.21.15.78:0/2961631294 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f77e00a2b90 0x7f77e00a5050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f77f4197ef0 tx=0x7f77e4005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:30.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.047+0000 7f77eaffd640 1 -- 172.21.15.78:0/2961631294 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f77f00bcad0 con 0x7f77f41006a0 2024-04-25T01:45:30.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.223+0000 7f77fbd63640 1 -- 172.21.15.78:0/2961631294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f77f40623e0 con 0x7f77f41006a0 2024-04-25T01:45:30.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.224+0000 7f77eaffd640 1 -- 172.21.15.78:0/2961631294 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f77f00bc3d0 con 0x7f77f41006a0 2024-04-25T01:45:30.216 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:45:30.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.226+0000 7f77fbd63640 1 -- 172.21.15.78:0/2961631294 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f77e00a2b90 msgr2=0x7f77e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.226+0000 7f77fbd63640 1 --2- 172.21.15.78:0/2961631294 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f77e00a2b90 0x7f77e00a5050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f77f4197ef0 tx=0x7f77e4005e50 comp rx=0 tx=0).stop 2024-04-25T01:45:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.226+0000 7f77fbd63640 1 -- 172.21.15.78:0/2961631294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77f41006a0 msgr2=0x7f77f41969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.226+0000 7f77fbd63640 1 --2- 172.21.15.78:0/2961631294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77f41006a0 0x7f77f41969e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f77f0031440 tx=0x7f77f0032e20 comp rx=0 tx=0).stop 2024-04-25T01:45:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.226+0000 7f77fbd63640 1 -- 172.21.15.78:0/2961631294 shutdown_connections 2024-04-25T01:45:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.226+0000 7f77fbd63640 1 --2- 172.21.15.78:0/2961631294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77f4102250 0x7f77f419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.226+0000 7f77fbd63640 1 --2- 172.21.15.78:0/2961631294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77f4101890 0x7f77f4196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.226+0000 7f77fbd63640 1 --2- 172.21.15.78:0/2961631294 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f77e00a2b90 0x7f77e00a5050 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.226+0000 7f77fbd63640 1 --2- 172.21.15.78:0/2961631294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77f41006a0 0x7f77f41969e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.226+0000 7f77fbd63640 1 -- 172.21.15.78:0/2961631294 >> 172.21.15.78:0/2961631294 conn(0x7f77f40fbc30 msgr2=0x7f77f4104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.226+0000 7f77fbd63640 1 -- 172.21.15.78:0/2961631294 shutdown_connections 2024-04-25T01:45:30.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.226+0000 7f77fbd63640 1 -- 172.21.15.78:0/2961631294 wait complete. 2024-04-25T01:45:30.228 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:30 smithi078 ceph-mon[29469]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 6.3 KiB/s, 1 objects/s recovering 2024-04-25T01:45:30.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:30 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1967251603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:45:30.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:30 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/993067183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:45:30.229 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:30 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3007571109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:45:30.230 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:45:30.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:30 smithi152 ceph-mon[32465]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 6.3 KiB/s, 1 objects/s recovering 2024-04-25T01:45:30.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:30 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1967251603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:45:30.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:30 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/993067183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:45:30.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:30 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3007571109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:45:30.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.334+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/4248317404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba40ffa60 msgr2=0x7f8ba40ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.334+0000 7f8ba8f8b640 1 --2- 172.21.15.78:0/4248317404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba40ffa60 0x7f8ba40ffe80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009930 tx=0x7f8b9402a420 comp rx=0 tx=0).stop 2024-04-25T01:45:30.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.334+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/4248317404 shutdown_connections 2024-04-25T01:45:30.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.334+0000 7f8ba8f8b640 1 --2- 172.21.15.78:0/4248317404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ba41033f0 0x7f8ba4109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.334+0000 7f8ba8f8b640 1 --2- 172.21.15.78:0/4248317404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba4102a30 0x7f8ba4102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.334+0000 7f8ba8f8b640 1 --2- 172.21.15.78:0/4248317404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba40ffa60 0x7f8ba40ffe80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.334+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/4248317404 >> 172.21.15.78:0/4248317404 conn(0x7f8ba40fb3b0 msgr2=0x7f8ba40fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:30.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.335+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/4248317404 shutdown_connections 2024-04-25T01:45:30.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.335+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/4248317404 wait complete. 2024-04-25T01:45:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.335+0000 7f8ba8f8b640 1 Processor -- start 2024-04-25T01:45:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.335+0000 7f8ba8f8b640 1 -- start start 2024-04-25T01:45:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.335+0000 7f8ba8f8b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba40ffa60 0x7f8ba4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.335+0000 7f8ba8f8b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ba4102a30 0x7f8ba4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.335+0000 7f8ba8f8b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba41033f0 0x7f8ba419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.335+0000 7f8ba8f8b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba410d430 con 0x7f8ba41033f0 2024-04-25T01:45:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.335+0000 7f8ba8f8b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba410d2b0 con 0x7f8ba4102a30 2024-04-25T01:45:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.335+0000 7f8ba8f8b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba410d5b0 con 0x7f8ba40ffa60 2024-04-25T01:45:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.335+0000 7f8ba2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba41033f0 0x7f8ba419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.335+0000 7f8ba2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba41033f0 0x7f8ba419f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60030/0 (socket says 172.21.15.78:60030) 2024-04-25T01:45:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.335+0000 7f8ba2d9d640 1 -- 172.21.15.78:0/3846883169 learned_addr learned my addr 172.21.15.78:0/3846883169 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.336+0000 7f8ba1d9b640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ba4102a30 0x7f8ba4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.336+0000 7f8ba259c640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba40ffa60 0x7f8ba4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.336+0000 7f8ba2d9d640 1 -- 172.21.15.78:0/3846883169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba40ffa60 msgr2=0x7f8ba4197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.336+0000 7f8ba2d9d640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba40ffa60 0x7f8ba4197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.336+0000 7f8ba2d9d640 1 -- 172.21.15.78:0/3846883169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ba4102a30 msgr2=0x7f8ba4198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.336+0000 7f8ba2d9d640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ba4102a30 0x7f8ba4198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.336+0000 7f8ba2d9d640 1 -- 172.21.15.78:0/3846883169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba419f730 con 0x7f8ba41033f0 2024-04-25T01:45:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.336+0000 7f8ba1d9b640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ba4102a30 0x7f8ba4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.336+0000 7f8ba2d9d640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba41033f0 0x7f8ba419f0c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00eaa0 tx=0x7f8b9c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.336+0000 7f8b937fe640 1 -- 172.21.15.78:0/3846883169 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c00cdb0 con 0x7f8ba41033f0 2024-04-25T01:45:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.336+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/3846883169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba419fa20 con 0x7f8ba41033f0 2024-04-25T01:45:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.336+0000 7f8b937fe640 1 -- 172.21.15.78:0/3846883169 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b9c005020 con 0x7f8ba41033f0 2024-04-25T01:45:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.336+0000 7f8b937fe640 1 -- 172.21.15.78:0/3846883169 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c01f060 con 0x7f8ba41033f0 2024-04-25T01:45:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.336+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/3846883169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba419ff20 con 0x7f8ba41033f0 2024-04-25T01:45:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.337+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/3846883169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba4108be0 con 0x7f8ba41033f0 2024-04-25T01:45:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.338+0000 7f8b937fe640 1 -- 172.21.15.78:0/3846883169 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8b9c005560 con 0x7f8ba41033f0 2024-04-25T01:45:30.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.339+0000 7f8b937fe640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8b8c0a2b90 0x7f8b8c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.339+0000 7f8ba259c640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8b8c0a2b90 0x7f8b8c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.339+0000 7f8b937fe640 1 -- 172.21.15.78:0/3846883169 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f8b9c0cf6b0 con 0x7f8ba41033f0 2024-04-25T01:45:30.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.339+0000 7f8ba259c640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8b8c0a2b90 0x7f8b8c0a5050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009900 tx=0x7f8b94000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:30.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.343+0000 7f8b937fe640 1 -- 172.21.15.78:0/3846883169 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8b9c016030 con 0x7f8ba41033f0 2024-04-25T01:45:30.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:30 smithi196 ceph-mon[32978]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 6.3 KiB/s, 1 objects/s recovering 2024-04-25T01:45:30.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:30 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1967251603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:45:30.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:30 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/993067183' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:45:30.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:30 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3007571109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:45:30.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.558+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/3846883169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ba402d170 con 0x7f8ba41033f0 2024-04-25T01:45:30.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.558+0000 7f8b937fe640 1 -- 172.21.15.78:0/3846883169 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f8b9c09c230 con 0x7f8ba41033f0 2024-04-25T01:45:30.554 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:30.554 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:43:17.355192+0000","modified":"2024-04-25T01:43:44.435289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24455,"mds_2":24448,"mds_3":24392,"mds_4":24379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1337519666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1337519666},{"type":"v1","addr":"172.21.15.78:6839","nonce":1337519666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1963410683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1963410683},{"type":"v1","addr":"172.21.15.78:6841","nonce":1963410683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24379":{"gid":24379,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1969344843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1969344843},{"type":"v1","addr":"172.21.15.152:6839","nonce":1969344843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24379,24491,24455,24392,24448]},"id":1}]} 2024-04-25T01:45:30.555 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-04-25T01:45:30.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.560+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/3846883169 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8b8c0a2b90 msgr2=0x7f8b8c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.560+0000 7f8ba8f8b640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8b8c0a2b90 0x7f8b8c0a5050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009900 tx=0x7f8b94000bc0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.560+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/3846883169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba41033f0 msgr2=0x7f8ba419f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.560+0000 7f8ba8f8b640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba41033f0 0x7f8ba419f0c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00eaa0 tx=0x7f8b9c00ef60 comp rx=0 tx=0).stop 2024-04-25T01:45:30.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.561+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/3846883169 shutdown_connections 2024-04-25T01:45:30.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.561+0000 7f8ba8f8b640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f8b8c0a2b90 0x7f8b8c0a5050 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.561+0000 7f8ba8f8b640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ba41033f0 0x7f8ba419f0c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.561+0000 7f8ba8f8b640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ba4102a30 0x7f8ba4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.561+0000 7f8ba8f8b640 1 --2- 172.21.15.78:0/3846883169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba40ffa60 0x7f8ba4197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.561+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/3846883169 >> 172.21.15.78:0/3846883169 conn(0x7f8ba40fb3b0 msgr2=0x7f8ba4105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:30.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.561+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/3846883169 shutdown_connections 2024-04-25T01:45:30.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.561+0000 7f8ba8f8b640 1 -- 172.21.15.78:0/3846883169 wait complete. 2024-04-25T01:45:30.565 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:45:30.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.670+0000 7f6d81fb6640 1 -- 172.21.15.78:0/3559529391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d7c102240 msgr2=0x7f6d7c108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.670+0000 7f6d81fb6640 1 --2- 172.21.15.78:0/3559529391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d7c102240 0x7f6d7c108ad0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009ca0 tx=0x7f6d7402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.670+0000 7f6d81fb6640 1 -- 172.21.15.78:0/3559529391 shutdown_connections 2024-04-25T01:45:30.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.670+0000 7f6d81fb6640 1 --2- 172.21.15.78:0/3559529391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d7c102240 0x7f6d7c108ad0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.670+0000 7f6d81fb6640 1 --2- 172.21.15.78:0/3559529391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d7c101880 0x7f6d7c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.670+0000 7f6d81fb6640 1 --2- 172.21.15.78:0/3559529391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d7c100690 0x7f6d7c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.670+0000 7f6d81fb6640 1 -- 172.21.15.78:0/3559529391 >> 172.21.15.78:0/3559529391 conn(0x7f6d7c0fbc40 msgr2=0x7f6d7c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.670+0000 7f6d81fb6640 1 -- 172.21.15.78:0/3559529391 shutdown_connections 2024-04-25T01:45:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.670+0000 7f6d81fb6640 1 -- 172.21.15.78:0/3559529391 wait complete. 2024-04-25T01:45:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d81fb6640 1 Processor -- start 2024-04-25T01:45:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d81fb6640 1 -- start start 2024-04-25T01:45:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d81fb6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d7c100690 0x7f6d7c196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d81fb6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d7c101880 0x7f6d7c196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d81fb6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d7c102240 0x7f6d7c19dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d81fb6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c10c380 con 0x7f6d7c102240 2024-04-25T01:45:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d81fb6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c10c200 con 0x7f6d7c100690 2024-04-25T01:45:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d81fb6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c10c500 con 0x7f6d7c101880 2024-04-25T01:45:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d7bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d7c102240 0x7f6d7c19dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d7bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d7c102240 0x7f6d7c19dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60042/0 (socket says 172.21.15.78:60042) 2024-04-25T01:45:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d7bfff640 1 -- 172.21.15.78:0/2849333111 learned_addr learned my addr 172.21.15.78:0/2849333111 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d7affd640 1 --2- 172.21.15.78:0/2849333111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d7c101880 0x7f6d7c196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d7b7fe640 1 --2- 172.21.15.78:0/2849333111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d7c100690 0x7f6d7c196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d7bfff640 1 -- 172.21.15.78:0/2849333111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d7c101880 msgr2=0x7f6d7c196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d7bfff640 1 --2- 172.21.15.78:0/2849333111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d7c101880 0x7f6d7c196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d7bfff640 1 -- 172.21.15.78:0/2849333111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d7c100690 msgr2=0x7f6d7c196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d7bfff640 1 --2- 172.21.15.78:0/2849333111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d7c100690 0x7f6d7c196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.671+0000 7f6d7bfff640 1 -- 172.21.15.78:0/2849333111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7c19e530 con 0x7f6d7c102240 2024-04-25T01:45:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.672+0000 7f6d7bfff640 1 --2- 172.21.15.78:0/2849333111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d7c102240 0x7f6d7c19dec0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6d740098d0 tx=0x7f6d7402f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.672+0000 7f6d78ff9640 1 -- 172.21.15.78:0/2849333111 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d74049070 con 0x7f6d7c102240 2024-04-25T01:45:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.672+0000 7f6d78ff9640 1 -- 172.21.15.78:0/2849333111 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d74031df0 con 0x7f6d7c102240 2024-04-25T01:45:30.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.672+0000 7f6d81fb6640 1 -- 172.21.15.78:0/2849333111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7c069de0 con 0x7f6d7c102240 2024-04-25T01:45:30.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.672+0000 7f6d78ff9640 1 -- 172.21.15.78:0/2849333111 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d740045d0 con 0x7f6d7c102240 2024-04-25T01:45:30.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.672+0000 7f6d81fb6640 1 -- 172.21.15.78:0/2849333111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d7c06a400 con 0x7f6d7c102240 2024-04-25T01:45:30.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.673+0000 7f6d81fb6640 1 -- 172.21.15.78:0/2849333111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d7c107350 con 0x7f6d7c102240 2024-04-25T01:45:30.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.674+0000 7f6d78ff9640 1 -- 172.21.15.78:0/2849333111 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6d7404c070 con 0x7f6d7c102240 2024-04-25T01:45:30.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.674+0000 7f6d78ff9640 1 --2- 172.21.15.78:0/2849333111 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6d640a2c10 0x7f6d640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.675+0000 7f6d7b7fe640 1 --2- 172.21.15.78:0/2849333111 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6d640a2c10 0x7f6d640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.675+0000 7f6d78ff9640 1 -- 172.21.15.78:0/2849333111 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f6d740f1e20 con 0x7f6d7c102240 2024-04-25T01:45:30.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.675+0000 7f6d7b7fe640 1 --2- 172.21.15.78:0/2849333111 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6d640a2c10 0x7f6d640a50d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c1016e0 tx=0x7f6d70008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:30.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.678+0000 7f6d78ff9640 1 -- 172.21.15.78:0/2849333111 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6d74080740 con 0x7f6d7c102240 2024-04-25T01:45:30.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.855+0000 7f6d81fb6640 1 -- 172.21.15.78:0/2849333111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d7c19e760 con 0x7f6d7c102240 2024-04-25T01:45:30.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.857+0000 7f6d78ff9640 1 -- 172.21.15.78:0/2849333111 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24844 (secure 0 0 0) 0x7f6d740be9a0 con 0x7f6d7c102240 2024-04-25T01:45:30.849 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:30.850 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":94,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:45:18.989934+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:45:30.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.859+0000 7f6d81fb6640 1 -- 172.21.15.78:0/2849333111 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6d640a2c10 msgr2=0x7f6d640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.859+0000 7f6d81fb6640 1 --2- 172.21.15.78:0/2849333111 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6d640a2c10 0x7f6d640a50d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c1016e0 tx=0x7f6d70008040 comp rx=0 tx=0).stop 2024-04-25T01:45:30.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.859+0000 7f6d81fb6640 1 -- 172.21.15.78:0/2849333111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d7c102240 msgr2=0x7f6d7c19dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.859+0000 7f6d81fb6640 1 --2- 172.21.15.78:0/2849333111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d7c102240 0x7f6d7c19dec0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6d740098d0 tx=0x7f6d7402f840 comp rx=0 tx=0).stop 2024-04-25T01:45:30.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.859+0000 7f6d81fb6640 1 -- 172.21.15.78:0/2849333111 shutdown_connections 2024-04-25T01:45:30.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.859+0000 7f6d81fb6640 1 --2- 172.21.15.78:0/2849333111 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f6d640a2c10 0x7f6d640a50d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.859+0000 7f6d81fb6640 1 --2- 172.21.15.78:0/2849333111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d7c102240 0x7f6d7c19dec0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.859+0000 7f6d81fb6640 1 --2- 172.21.15.78:0/2849333111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d7c101880 0x7f6d7c196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.859+0000 7f6d81fb6640 1 --2- 172.21.15.78:0/2849333111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d7c100690 0x7f6d7c196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.859+0000 7f6d81fb6640 1 -- 172.21.15.78:0/2849333111 >> 172.21.15.78:0/2849333111 conn(0x7f6d7c0fbc40 msgr2=0x7f6d7c104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:30.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.859+0000 7f6d81fb6640 1 -- 172.21.15.78:0/2849333111 shutdown_connections 2024-04-25T01:45:30.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.859+0000 7f6d81fb6640 1 -- 172.21.15.78:0/2849333111 wait complete. 2024-04-25T01:45:30.864 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:38:39.136418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:43:15.613881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:43:16.600270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:43:19.650457+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:45:30.864 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:45:30.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 -- 172.21.15.78:0/1882960792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3854103340 msgr2=0x7f3854109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 --2- 172.21.15.78:0/1882960792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3854103340 0x7f3854109bd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f384c009ca0 tx=0x7f384c02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 -- 172.21.15.78:0/1882960792 shutdown_connections 2024-04-25T01:45:30.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 --2- 172.21.15.78:0/1882960792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3854103340 0x7f3854109bd0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 --2- 172.21.15.78:0/1882960792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3854102980 0x7f3854102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 --2- 172.21.15.78:0/1882960792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3854101790 0x7f3854101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 -- 172.21.15.78:0/1882960792 >> 172.21.15.78:0/1882960792 conn(0x7f38540fcd20 msgr2=0x7f38540ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:30.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 -- 172.21.15.78:0/1882960792 shutdown_connections 2024-04-25T01:45:30.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 -- 172.21.15.78:0/1882960792 wait complete. 2024-04-25T01:45:30.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 Processor -- start 2024-04-25T01:45:30.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 -- start start 2024-04-25T01:45:30.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3854101790 0x7f3854197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3854102980 0x7f3854197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3854103340 0x7f385419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f385410d410 con 0x7f3854103340 2024-04-25T01:45:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f385410d290 con 0x7f3854101790 2024-04-25T01:45:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.973+0000 7f385afb7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f385410d590 con 0x7f3854102980 2024-04-25T01:45:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3859554640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3854103340 0x7f385419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3859554640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3854103340 0x7f385419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60068/0 (socket says 172.21.15.78:60068) 2024-04-25T01:45:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3859554640 1 -- 172.21.15.78:0/241950029 learned_addr learned my addr 172.21.15.78:0/241950029 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3858d53640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3854101790 0x7f3854197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3853fff640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3854102980 0x7f3854197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3859554640 1 -- 172.21.15.78:0/241950029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3854102980 msgr2=0x7f3854197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3859554640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3854102980 0x7f3854197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3859554640 1 -- 172.21.15.78:0/241950029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3854101790 msgr2=0x7f3854197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3859554640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3854101790 0x7f3854197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3859554640 1 -- 172.21.15.78:0/241950029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385419f670 con 0x7f3854103340 2024-04-25T01:45:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3858d53640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3854101790 0x7f3854197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3859554640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3854103340 0x7f385419f000 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f384c0098d0 tx=0x7f384c02f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:30.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3851ffb640 1 -- 172.21.15.78:0/241950029 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c02fc30 con 0x7f3854103340 2024-04-25T01:45:30.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3851ffb640 1 -- 172.21.15.78:0/241950029 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f384c003e50 con 0x7f3854103340 2024-04-25T01:45:30.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f385afb7640 1 -- 172.21.15.78:0/241950029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385419f900 con 0x7f3854103340 2024-04-25T01:45:30.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f3851ffb640 1 -- 172.21.15.78:0/241950029 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c0305c0 con 0x7f3854103340 2024-04-25T01:45:30.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.974+0000 7f385afb7640 1 -- 172.21.15.78:0/241950029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385419fea0 con 0x7f3854103340 2024-04-25T01:45:30.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.975+0000 7f385afb7640 1 -- 172.21.15.78:0/241950029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3854108ab0 con 0x7f3854103340 2024-04-25T01:45:30.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.977+0000 7f3851ffb640 1 -- 172.21.15.78:0/241950029 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f384c003970 con 0x7f3854103340 2024-04-25T01:45:30.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.977+0000 7f3851ffb640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f38440a2c10 0x7f38440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:30.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.977+0000 7f3858d53640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f38440a2c10 0x7f38440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:30.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.978+0000 7f3851ffb640 1 -- 172.21.15.78:0/241950029 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f384c0f0880 con 0x7f3854103340 2024-04-25T01:45:30.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.978+0000 7f3858d53640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f38440a2c10 0x7f38440a50d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f38480058c0 tx=0x7f3848005ec0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:30.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:30.982+0000 7f3851ffb640 1 -- 172.21.15.78:0/241950029 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f384c0bd400 con 0x7f3854103340 2024-04-25T01:45:31.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.159+0000 7f385afb7640 1 -- 172.21.15.78:0/241950029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3854062340 con 0x7f3854103340 2024-04-25T01:45:31.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.160+0000 7f3851ffb640 1 -- 172.21.15.78:0/241950029 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f384c0bcd00 con 0x7f3854103340 2024-04-25T01:45:31.152 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T01:45:31.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.162+0000 7f385afb7640 1 -- 172.21.15.78:0/241950029 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f38440a2c10 msgr2=0x7f38440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.162+0000 7f385afb7640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f38440a2c10 0x7f38440a50d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f38480058c0 tx=0x7f3848005ec0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.162+0000 7f385afb7640 1 -- 172.21.15.78:0/241950029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3854103340 msgr2=0x7f385419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.162+0000 7f385afb7640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3854103340 0x7f385419f000 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f384c0098d0 tx=0x7f384c02f840 comp rx=0 tx=0).stop 2024-04-25T01:45:31.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.162+0000 7f385afb7640 1 -- 172.21.15.78:0/241950029 shutdown_connections 2024-04-25T01:45:31.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.162+0000 7f385afb7640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f38440a2c10 0x7f38440a50d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.162+0000 7f385afb7640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3854103340 0x7f385419f000 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.162+0000 7f385afb7640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3854102980 0x7f3854197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.162+0000 7f385afb7640 1 --2- 172.21.15.78:0/241950029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3854101790 0x7f3854197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.162+0000 7f385afb7640 1 -- 172.21.15.78:0/241950029 >> 172.21.15.78:0/241950029 conn(0x7f38540fcd20 msgr2=0x7f3854105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:31.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.162+0000 7f385afb7640 1 -- 172.21.15.78:0/241950029 shutdown_connections 2024-04-25T01:45:31.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.162+0000 7f385afb7640 1 -- 172.21.15.78:0/241950029 wait complete. 2024-04-25T01:45:31.165 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:31 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2961631294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:45:31.165 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:31 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3846883169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:31.165 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:31 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2849333111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:31.166 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:45:31.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.275+0000 7f292199e640 1 -- 172.21.15.78:0/429708424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c1020f0 msgr2=0x7f291c108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.275+0000 7f292199e640 1 --2- 172.21.15.78:0/429708424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c1020f0 0x7f291c108980 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2914009ca0 tx=0x7f291402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.275+0000 7f292199e640 1 -- 172.21.15.78:0/429708424 shutdown_connections 2024-04-25T01:45:31.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.275+0000 7f292199e640 1 --2- 172.21.15.78:0/429708424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c1020f0 0x7f291c108980 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.275+0000 7f292199e640 1 --2- 172.21.15.78:0/429708424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f291c101730 0x7f291c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.275+0000 7f292199e640 1 --2- 172.21.15.78:0/429708424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f291c100540 0x7f291c100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.275+0000 7f292199e640 1 -- 172.21.15.78:0/429708424 >> 172.21.15.78:0/429708424 conn(0x7f291c0fbaf0 msgr2=0x7f291c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:31.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.275+0000 7f292199e640 1 -- 172.21.15.78:0/429708424 shutdown_connections 2024-04-25T01:45:31.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.275+0000 7f292199e640 1 -- 172.21.15.78:0/429708424 wait complete. 2024-04-25T01:45:31.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292199e640 1 Processor -- start 2024-04-25T01:45:31.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292199e640 1 -- start start 2024-04-25T01:45:31.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292199e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c100540 0x7f291c196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:31.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292199e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f291c101730 0x7f291c196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:31.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292199e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f291c1020f0 0x7f291c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:31.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292199e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f291c10c230 con 0x7f291c100540 2024-04-25T01:45:31.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292199e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f291c10c0b0 con 0x7f291c1020f0 2024-04-25T01:45:31.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292199e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f291c10c3b0 con 0x7f291c101730 2024-04-25T01:45:31.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292099c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c100540 0x7f291c196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:31.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292099c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c100540 0x7f291c196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60086/0 (socket says 172.21.15.78:60086) 2024-04-25T01:45:31.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292099c640 1 -- 172.21.15.78:0/3506428519 learned_addr learned my addr 172.21.15.78:0/3506428519 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292119d640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f291c1020f0 0x7f291c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f2913fff640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f291c101730 0x7f291c196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292099c640 1 -- 172.21.15.78:0/3506428519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f291c101730 msgr2=0x7f291c196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292099c640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f291c101730 0x7f291c196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292099c640 1 -- 172.21.15.78:0/3506428519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f291c1020f0 msgr2=0x7f291c19de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292099c640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f291c1020f0 0x7f291c19de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292099c640 1 -- 172.21.15.78:0/3506428519 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c19e490 con 0x7f291c100540 2024-04-25T01:45:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292119d640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f291c1020f0 0x7f291c19de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292099c640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c100540 0x7f291c196870 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f291800b450 tx=0x7f291800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f2911ffb640 1 -- 172.21.15.78:0/3506428519 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918015020 con 0x7f291c100540 2024-04-25T01:45:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.276+0000 7f292199e640 1 -- 172.21.15.78:0/3506428519 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c19e780 con 0x7f291c100540 2024-04-25T01:45:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.277+0000 7f2911ffb640 1 -- 172.21.15.78:0/3506428519 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2918011d70 con 0x7f291c100540 2024-04-25T01:45:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.277+0000 7f2911ffb640 1 -- 172.21.15.78:0/3506428519 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291800f340 con 0x7f291c100540 2024-04-25T01:45:31.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.277+0000 7f292199e640 1 -- 172.21.15.78:0/3506428519 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c19eda0 con 0x7f291c100540 2024-04-25T01:45:31.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.278+0000 7f292199e640 1 -- 172.21.15.78:0/3506428519 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291c107800 con 0x7f291c100540 2024-04-25T01:45:31.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.279+0000 7f2911ffb640 1 -- 172.21.15.78:0/3506428519 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2918018020 con 0x7f291c100540 2024-04-25T01:45:31.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.279+0000 7f2911ffb640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29040a2b90 0x7f29040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:31.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.279+0000 7f2913fff640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29040a2b90 0x7f29040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:31.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.279+0000 7f2911ffb640 1 -- 172.21.15.78:0/3506428519 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f29180cef80 con 0x7f291c100540 2024-04-25T01:45:31.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.280+0000 7f2913fff640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29040a2b90 0x7f29040a5050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f291c197d80 tx=0x7f290c00a4c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:31.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.284+0000 7f2911ffb640 1 -- 172.21.15.78:0/3506428519 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f291801f020 con 0x7f291c100540 2024-04-25T01:45:31.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:31 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2961631294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:45:31.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:31 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3846883169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:31.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:31 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2849333111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:31.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:31 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2961631294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:45:31.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:31 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3846883169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:31.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:31 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2849333111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:31.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.459+0000 7f292199e640 1 -- 172.21.15.78:0/3506428519 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f291c062340 con 0x7f291c100540 2024-04-25T01:45:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.459+0000 7f2911ffb640 1 -- 172.21.15.78:0/3506428519 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f291800d040 con 0x7f291c100540 2024-04-25T01:45:31.452 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:45:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.461+0000 7f292199e640 1 -- 172.21.15.78:0/3506428519 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29040a2b90 msgr2=0x7f29040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.461+0000 7f292199e640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29040a2b90 0x7f29040a5050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f291c197d80 tx=0x7f290c00a4c0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.461+0000 7f292199e640 1 -- 172.21.15.78:0/3506428519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c100540 msgr2=0x7f291c196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.461+0000 7f292199e640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c100540 0x7f291c196870 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f291800b450 tx=0x7f291800b910 comp rx=0 tx=0).stop 2024-04-25T01:45:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.461+0000 7f292199e640 1 -- 172.21.15.78:0/3506428519 shutdown_connections 2024-04-25T01:45:31.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.461+0000 7f292199e640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f291c1020f0 0x7f291c19de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.461+0000 7f292199e640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f291c101730 0x7f291c196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.461+0000 7f292199e640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f29040a2b90 0x7f29040a5050 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.461+0000 7f292199e640 1 --2- 172.21.15.78:0/3506428519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f291c100540 0x7f291c196870 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.461+0000 7f292199e640 1 -- 172.21.15.78:0/3506428519 >> 172.21.15.78:0/3506428519 conn(0x7f291c0fbaf0 msgr2=0x7f291c1049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:31.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.461+0000 7f292199e640 1 -- 172.21.15.78:0/3506428519 shutdown_connections 2024-04-25T01:45:31.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.461+0000 7f292199e640 1 -- 172.21.15.78:0/3506428519 wait complete. 2024-04-25T01:45:31.465 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:45:31.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.573+0000 7fb212916640 1 -- 172.21.15.78:0/351069025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb20c100680 msgr2=0x7fb20c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.573+0000 7fb212916640 1 --2- 172.21.15.78:0/351069025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb20c100680 0x7fb20c100aa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009900 tx=0x7fb1fc02a410 comp rx=0 tx=0).stop 2024-04-25T01:45:31.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.573+0000 7fb212916640 1 -- 172.21.15.78:0/351069025 shutdown_connections 2024-04-25T01:45:31.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.573+0000 7fb212916640 1 --2- 172.21.15.78:0/351069025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb20c102230 0x7fb20c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.573+0000 7fb212916640 1 --2- 172.21.15.78:0/351069025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb20c101870 0x7fb20c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.573+0000 7fb212916640 1 --2- 172.21.15.78:0/351069025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb20c100680 0x7fb20c100aa0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.573+0000 7fb212916640 1 -- 172.21.15.78:0/351069025 >> 172.21.15.78:0/351069025 conn(0x7fb20c0fbbf0 msgr2=0x7fb20c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.573+0000 7fb212916640 1 -- 172.21.15.78:0/351069025 shutdown_connections 2024-04-25T01:45:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.573+0000 7fb212916640 1 -- 172.21.15.78:0/351069025 wait complete. 2024-04-25T01:45:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb212916640 1 Processor -- start 2024-04-25T01:45:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb212916640 1 -- start start 2024-04-25T01:45:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb212916640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb20c100680 0x7fb20c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb212916640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb20c101870 0x7fb20c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb212916640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb20c102230 0x7fb20c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb212916640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c10c300 con 0x7fb20c100680 2024-04-25T01:45:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb212916640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c10c180 con 0x7fb20c101870 2024-04-25T01:45:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb212916640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c10c480 con 0x7fb20c102230 2024-04-25T01:45:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb20bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb20c100680 0x7fb20c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb20bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb20c100680 0x7fb20c069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60096/0 (socket says 172.21.15.78:60096) 2024-04-25T01:45:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb20bfff640 1 -- 172.21.15.78:0/2034468622 learned_addr learned my addr 172.21.15.78:0/2034468622 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb210eb3640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb20c102230 0x7fb20c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb20b7fe640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb20c101870 0x7fb20c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb20bfff640 1 -- 172.21.15.78:0/2034468622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb20c102230 msgr2=0x7fb20c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb20bfff640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb20c102230 0x7fb20c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb20bfff640 1 -- 172.21.15.78:0/2034468622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb20c101870 msgr2=0x7fb20c19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb20bfff640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb20c101870 0x7fb20c19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.574+0000 7fb20bfff640 1 -- 172.21.15.78:0/2034468622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c19f7b0 con 0x7fb20c100680 2024-04-25T01:45:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.575+0000 7fb20b7fe640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb20c101870 0x7fb20c19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.575+0000 7fb210eb3640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb20c102230 0x7fb20c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.575+0000 7fb20bfff640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb20c100680 0x7fb20c069440 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc031610 tx=0x7fb1fc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:31.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.575+0000 7fb2097fa640 1 -- 172.21.15.78:0/2034468622 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc03f030 con 0x7fb20c100680 2024-04-25T01:45:31.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.575+0000 7fb2097fa640 1 -- 172.21.15.78:0/2034468622 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1fc003ce0 con 0x7fb20c100680 2024-04-25T01:45:31.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.575+0000 7fb212916640 1 -- 172.21.15.78:0/2034468622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20c19f9e0 con 0x7fb20c100680 2024-04-25T01:45:31.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.575+0000 7fb2097fa640 1 -- 172.21.15.78:0/2034468622 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc035070 con 0x7fb20c100680 2024-04-25T01:45:31.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.575+0000 7fb212916640 1 -- 172.21.15.78:0/2034468622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20c19fed0 con 0x7fb20c100680 2024-04-25T01:45:31.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.576+0000 7fb212916640 1 -- 172.21.15.78:0/2034468622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20c107a10 con 0x7fb20c100680 2024-04-25T01:45:31.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.577+0000 7fb2097fa640 1 -- 172.21.15.78:0/2034468622 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb1fc042030 con 0x7fb20c100680 2024-04-25T01:45:31.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.577+0000 7fb2097fa640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb1f40a2b90 0x7fb1f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:31.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.577+0000 7fb20b7fe640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb1f40a2b90 0x7fb1f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:31.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.577+0000 7fb2097fa640 1 -- 172.21.15.78:0/2034468622 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fb1fc0ef740 con 0x7fb20c100680 2024-04-25T01:45:31.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.578+0000 7fb20b7fe640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb1f40a2b90 0x7fb1f40a5050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0691b0 tx=0x7fb200005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:31.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.582+0000 7fb2097fa640 1 -- 172.21.15.78:0/2034468622 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb1fc0bc2c0 con 0x7fb20c100680 2024-04-25T01:45:31.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.757+0000 7fb212916640 1 -- 172.21.15.78:0/2034468622 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb20c0623e0 con 0x7fb20c100680 2024-04-25T01:45:31.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.758+0000 7fb2097fa640 1 -- 172.21.15.78:0/2034468622 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fb1fc0bbbc0 con 0x7fb20c100680 2024-04-25T01:45:31.750 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T01:45:31.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.760+0000 7fb212916640 1 -- 172.21.15.78:0/2034468622 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb1f40a2b90 msgr2=0x7fb1f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.760+0000 7fb212916640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb1f40a2b90 0x7fb1f40a5050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0691b0 tx=0x7fb200005e50 comp rx=0 tx=0).stop 2024-04-25T01:45:31.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.760+0000 7fb212916640 1 -- 172.21.15.78:0/2034468622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb20c100680 msgr2=0x7fb20c069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.760+0000 7fb212916640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb20c100680 0x7fb20c069440 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc031610 tx=0x7fb1fc0043b0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.760+0000 7fb212916640 1 -- 172.21.15.78:0/2034468622 shutdown_connections 2024-04-25T01:45:31.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.760+0000 7fb212916640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb20c102230 0x7fb20c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.760+0000 7fb212916640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb20c101870 0x7fb20c19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.760+0000 7fb212916640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb1f40a2b90 0x7fb1f40a5050 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.760+0000 7fb212916640 1 --2- 172.21.15.78:0/2034468622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb20c100680 0x7fb20c069440 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.760+0000 7fb212916640 1 -- 172.21.15.78:0/2034468622 >> 172.21.15.78:0/2034468622 conn(0x7fb20c0fbbf0 msgr2=0x7fb20c104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:31.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.760+0000 7fb212916640 1 -- 172.21.15.78:0/2034468622 shutdown_connections 2024-04-25T01:45:31.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.760+0000 7fb212916640 1 -- 172.21.15.78:0/2034468622 wait complete. 2024-04-25T01:45:31.763 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:45:31.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.872+0000 7f2ece102640 1 -- 172.21.15.78:0/3573116282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec8101870 msgr2=0x7f2ec8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.872+0000 7f2ece102640 1 --2- 172.21.15.78:0/3573116282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec8101870 0x7f2ec8101cf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009900 tx=0x7f2eb002a410 comp rx=0 tx=0).stop 2024-04-25T01:45:31.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.872+0000 7f2ece102640 1 -- 172.21.15.78:0/3573116282 shutdown_connections 2024-04-25T01:45:31.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.872+0000 7f2ece102640 1 --2- 172.21.15.78:0/3573116282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec8102230 0x7f2ec8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.872+0000 7f2ece102640 1 --2- 172.21.15.78:0/3573116282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec8101870 0x7f2ec8101cf0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.872+0000 7f2ece102640 1 --2- 172.21.15.78:0/3573116282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ec8100680 0x7f2ec8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.872+0000 7f2ece102640 1 -- 172.21.15.78:0/3573116282 >> 172.21.15.78:0/3573116282 conn(0x7f2ec80fbbf0 msgr2=0x7f2ec80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ece102640 1 -- 172.21.15.78:0/3573116282 shutdown_connections 2024-04-25T01:45:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ece102640 1 -- 172.21.15.78:0/3573116282 wait complete. 2024-04-25T01:45:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ece102640 1 Processor -- start 2024-04-25T01:45:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ece102640 1 -- start start 2024-04-25T01:45:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ece102640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ec8100680 0x7f2ec8197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ece102640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec8101870 0x7f2ec81981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ece102640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec8102230 0x7f2ec819f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ece102640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec810c440 con 0x7f2ec8102230 2024-04-25T01:45:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ece102640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec810c2c0 con 0x7f2ec8100680 2024-04-25T01:45:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ece102640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec810c5c0 con 0x7f2ec8101870 2024-04-25T01:45:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ec7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec8102230 0x7f2ec819f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ec7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec8102230 0x7f2ec819f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60106/0 (socket says 172.21.15.78:60106) 2024-04-25T01:45:31.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ec7fff640 1 -- 172.21.15.78:0/406455652 learned_addr learned my addr 172.21.15.78:0/406455652 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:31.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ebffff640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec8101870 0x7f2ec81981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:31.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.873+0000 7f2ec77fe640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ec8100680 0x7f2ec8197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:31.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.874+0000 7f2ec7fff640 1 -- 172.21.15.78:0/406455652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec8101870 msgr2=0x7f2ec81981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.874+0000 7f2ec7fff640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec8101870 0x7f2ec81981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.874+0000 7f2ec7fff640 1 -- 172.21.15.78:0/406455652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ec8100680 msgr2=0x7f2ec8197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:31.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.874+0000 7f2ec7fff640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ec8100680 0x7f2ec8197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:31.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.874+0000 7f2ec7fff640 1 -- 172.21.15.78:0/406455652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec819f8d0 con 0x7f2ec8102230 2024-04-25T01:45:31.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.874+0000 7f2ec77fe640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ec8100680 0x7f2ec8197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:31.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.874+0000 7f2ec7fff640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec8102230 0x7f2ec819f230 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000b970 tx=0x7f2ec000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.874+0000 7f2ec57fa640 1 -- 172.21.15.78:0/406455652 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0019070 con 0x7f2ec8102230 2024-04-25T01:45:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.874+0000 7f2ec57fa640 1 -- 172.21.15.78:0/406455652 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec000cb40 con 0x7f2ec8102230 2024-04-25T01:45:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.874+0000 7f2ece102640 1 -- 172.21.15.78:0/406455652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec819fbc0 con 0x7f2ec8102230 2024-04-25T01:45:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.874+0000 7f2ec57fa640 1 -- 172.21.15.78:0/406455652 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0007650 con 0x7f2ec8102230 2024-04-25T01:45:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.874+0000 7f2ece102640 1 -- 172.21.15.78:0/406455652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec81a0160 con 0x7f2ec8102230 2024-04-25T01:45:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.875+0000 7f2ece102640 1 -- 172.21.15.78:0/406455652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec81a0390 con 0x7f2ec8102230 2024-04-25T01:45:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.876+0000 7f2ec57fa640 1 -- 172.21.15.78:0/406455652 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ec001c070 con 0x7f2ec8102230 2024-04-25T01:45:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.876+0000 7f2ec57fa640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2eb40a2b90 0x7f2eb40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.877+0000 7f2ec77fe640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2eb40a2b90 0x7f2eb40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.877+0000 7f2ec57fa640 1 -- 172.21.15.78:0/406455652 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f2ec00ce9b0 con 0x7f2ec8102230 2024-04-25T01:45:31.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.877+0000 7f2ec77fe640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2eb40a2b90 0x7f2eb40a5050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8198b60 tx=0x7f2eb8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:31.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:31.881+0000 7f2ec57fa640 1 -- 172.21.15.78:0/406455652 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2ec009aee0 con 0x7f2ec8102230 2024-04-25T01:45:32.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.059+0000 7f2ece102640 1 -- 172.21.15.78:0/406455652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2ec81a0600 con 0x7f2ec8102230 2024-04-25T01:45:32.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.060+0000 7f2ec57fa640 1 -- 172.21.15.78:0/406455652 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f2ec009f1e0 con 0x7f2ec8102230 2024-04-25T01:45:32.052 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:45:32.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.062+0000 7f2ece102640 1 -- 172.21.15.78:0/406455652 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2eb40a2b90 msgr2=0x7f2eb40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.062+0000 7f2ece102640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2eb40a2b90 0x7f2eb40a5050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8198b60 tx=0x7f2eb8008040 comp rx=0 tx=0).stop 2024-04-25T01:45:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.062+0000 7f2ece102640 1 -- 172.21.15.78:0/406455652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec8102230 msgr2=0x7f2ec819f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.062+0000 7f2ece102640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec8102230 0x7f2ec819f230 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000b970 tx=0x7f2ec000be30 comp rx=0 tx=0).stop 2024-04-25T01:45:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.062+0000 7f2ece102640 1 -- 172.21.15.78:0/406455652 shutdown_connections 2024-04-25T01:45:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.062+0000 7f2ece102640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f2eb40a2b90 0x7f2eb40a5050 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.062+0000 7f2ece102640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ec8102230 0x7f2ec819f230 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.062+0000 7f2ece102640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec8101870 0x7f2ec81981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.062+0000 7f2ece102640 1 --2- 172.21.15.78:0/406455652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ec8100680 0x7f2ec8197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.062+0000 7f2ece102640 1 -- 172.21.15.78:0/406455652 >> 172.21.15.78:0/406455652 conn(0x7f2ec80fbbf0 msgr2=0x7f2ec80fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.062+0000 7f2ece102640 1 -- 172.21.15.78:0/406455652 shutdown_connections 2024-04-25T01:45:32.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.062+0000 7f2ece102640 1 -- 172.21.15.78:0/406455652 wait complete. 2024-04-25T01:45:32.066 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:45:32.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.172+0000 7f668ac15640 1 -- 172.21.15.78:0/131889752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6684101880 msgr2=0x7f6684101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.172+0000 7f668ac15640 1 --2- 172.21.15.78:0/131889752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6684101880 0x7f6684101d00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6678009900 tx=0x7f667802a410 comp rx=0 tx=0).stop 2024-04-25T01:45:32.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.172+0000 7f668ac15640 1 -- 172.21.15.78:0/131889752 shutdown_connections 2024-04-25T01:45:32.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.172+0000 7f668ac15640 1 --2- 172.21.15.78:0/131889752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6684102240 0x7f6684108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.172+0000 7f668ac15640 1 --2- 172.21.15.78:0/131889752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6684101880 0x7f6684101d00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.172+0000 7f668ac15640 1 --2- 172.21.15.78:0/131889752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6684100690 0x7f6684100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.172+0000 7f668ac15640 1 -- 172.21.15.78:0/131889752 >> 172.21.15.78:0/131889752 conn(0x7f66840fbc20 msgr2=0x7f66840fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.172+0000 7f668ac15640 1 -- 172.21.15.78:0/131889752 shutdown_connections 2024-04-25T01:45:32.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.172+0000 7f668ac15640 1 -- 172.21.15.78:0/131889752 wait complete. 2024-04-25T01:45:32.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.172+0000 7f668ac15640 1 Processor -- start 2024-04-25T01:45:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.172+0000 7f668ac15640 1 -- start start 2024-04-25T01:45:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f668ac15640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6684100690 0x7f6684196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f668ac15640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6684101880 0x7f6684196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f668ac15640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6684102240 0x7f668419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f668ac15640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f668410c2f0 con 0x7f6684102240 2024-04-25T01:45:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f668ac15640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f668410c170 con 0x7f6684100690 2024-04-25T01:45:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f668ac15640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f668410c470 con 0x7f6684101880 2024-04-25T01:45:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f66891b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6684102240 0x7f668419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f66891b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6684102240 0x7f668419df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60132/0 (socket says 172.21.15.78:60132) 2024-04-25T01:45:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f66891b2640 1 -- 172.21.15.78:0/2095318100 learned_addr learned my addr 172.21.15.78:0/2095318100 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f6683fff640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6684101880 0x7f6684196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f66889b1640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6684100690 0x7f6684196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f66891b2640 1 -- 172.21.15.78:0/2095318100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6684101880 msgr2=0x7f6684196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f66891b2640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6684101880 0x7f6684196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f66891b2640 1 -- 172.21.15.78:0/2095318100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6684100690 msgr2=0x7f6684196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f66891b2640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6684100690 0x7f6684196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f66891b2640 1 -- 172.21.15.78:0/2095318100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668419e5b0 con 0x7f6684102240 2024-04-25T01:45:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f66889b1640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6684100690 0x7f6684196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f66891b2640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6684102240 0x7f668419df40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6674007850 tx=0x7f6674007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f6681ffb640 1 -- 172.21.15.78:0/2095318100 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674019020 con 0x7f6684102240 2024-04-25T01:45:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f668ac15640 1 -- 172.21.15.78:0/2095318100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6684069e00 con 0x7f6684102240 2024-04-25T01:45:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f6681ffb640 1 -- 172.21.15.78:0/2095318100 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6674005500 con 0x7f6684102240 2024-04-25T01:45:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.173+0000 7f6681ffb640 1 -- 172.21.15.78:0/2095318100 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667400e890 con 0x7f6684102240 2024-04-25T01:45:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.174+0000 7f668ac15640 1 -- 172.21.15.78:0/2095318100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668406a320 con 0x7f6684102240 2024-04-25T01:45:32.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.174+0000 7f668ac15640 1 -- 172.21.15.78:0/2095318100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66840689b0 con 0x7f6684102240 2024-04-25T01:45:32.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.175+0000 7f6681ffb640 1 -- 172.21.15.78:0/2095318100 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f66740046c0 con 0x7f6684102240 2024-04-25T01:45:32.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.176+0000 7f6681ffb640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f666c0a2b90 0x7f666c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.176+0000 7f66889b1640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f666c0a2b90 0x7f666c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.176+0000 7f6681ffb640 1 -- 172.21.15.78:0/2095318100 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f66740d0a70 con 0x7f6684102240 2024-04-25T01:45:32.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.176+0000 7f66889b1640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f666c0a2b90 0x7f666c0a5050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f66841016e0 tx=0x7f667c008370 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.180+0000 7f6681ffb640 1 -- 172.21.15.78:0/2095318100 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6674023020 con 0x7f6684102240 2024-04-25T01:45:32.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:32 smithi152 ceph-mon[32465]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 5.3 KiB/s, 1 objects/s recovering 2024-04-25T01:45:32.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:32 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/241950029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:45:32.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:32 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3506428519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:45:32.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:32 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2034468622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:45:32.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:32 smithi196 ceph-mon[32978]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 5.3 KiB/s, 1 objects/s recovering 2024-04-25T01:45:32.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:32 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/241950029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:45:32.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:32 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3506428519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:45:32.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:32 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2034468622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:45:32.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.391+0000 7f668ac15640 1 -- 172.21.15.78:0/2095318100 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f668419e840 con 0x7f6684102240 2024-04-25T01:45:32.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.392+0000 7f6681ffb640 1 -- 172.21.15.78:0/2095318100 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f667409d680 con 0x7f6684102240 2024-04-25T01:45:32.384 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:32.385 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:43:17.355192+0000","modified":"2024-04-25T01:43:44.435289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24455,"mds_2":24448,"mds_3":24392,"mds_4":24379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1337519666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1337519666},{"type":"v1","addr":"172.21.15.78:6839","nonce":1337519666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1963410683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1963410683},{"type":"v1","addr":"172.21.15.78:6841","nonce":1963410683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24379":{"gid":24379,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1969344843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1969344843},{"type":"v1","addr":"172.21.15.152:6839","nonce":1969344843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24379,24491,24455,24392,24448]},"id":1}]} 2024-04-25T01:45:32.385 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-04-25T01:45:32.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.394+0000 7f668ac15640 1 -- 172.21.15.78:0/2095318100 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f666c0a2b90 msgr2=0x7f666c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.394+0000 7f668ac15640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f666c0a2b90 0x7f666c0a5050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f66841016e0 tx=0x7f667c008370 comp rx=0 tx=0).stop 2024-04-25T01:45:32.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.394+0000 7f668ac15640 1 -- 172.21.15.78:0/2095318100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6684102240 msgr2=0x7f668419df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.394+0000 7f668ac15640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6684102240 0x7f668419df40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6674007850 tx=0x7f6674007d10 comp rx=0 tx=0).stop 2024-04-25T01:45:32.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.394+0000 7f668ac15640 1 -- 172.21.15.78:0/2095318100 shutdown_connections 2024-04-25T01:45:32.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.394+0000 7f668ac15640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f666c0a2b90 0x7f666c0a5050 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.394+0000 7f668ac15640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6684102240 0x7f668419df40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.394+0000 7f668ac15640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6684101880 0x7f6684196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.394+0000 7f668ac15640 1 --2- 172.21.15.78:0/2095318100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6684100690 0x7f6684196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.394+0000 7f668ac15640 1 -- 172.21.15.78:0/2095318100 >> 172.21.15.78:0/2095318100 conn(0x7f66840fbc20 msgr2=0x7f6684104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.394+0000 7f668ac15640 1 -- 172.21.15.78:0/2095318100 shutdown_connections 2024-04-25T01:45:32.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.394+0000 7f668ac15640 1 -- 172.21.15.78:0/2095318100 wait complete. 2024-04-25T01:45:32.396 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:32 smithi078 ceph-mon[29469]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 5.3 KiB/s, 1 objects/s recovering 2024-04-25T01:45:32.397 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:32 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/241950029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:45:32.397 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:32 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3506428519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:45:32.397 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:32 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2034468622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:45:32.398 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:45:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.503+0000 7f517d4ba640 1 -- 172.21.15.78:0/51422153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5178101c30 msgr2=0x7f5178102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.503+0000 7f517d4ba640 1 --2- 172.21.15.78:0/51422153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5178101c30 0x7f5178102050 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5168009900 tx=0x7f516802a410 comp rx=0 tx=0).stop 2024-04-25T01:45:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.503+0000 7f517d4ba640 1 -- 172.21.15.78:0/51422153 shutdown_connections 2024-04-25T01:45:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.503+0000 7f517d4ba640 1 --2- 172.21.15.78:0/51422153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5178101320 0x7f5178108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.503+0000 7f517d4ba640 1 --2- 172.21.15.78:0/51422153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5178100960 0x7f5178100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.503+0000 7f517d4ba640 1 --2- 172.21.15.78:0/51422153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5178101c30 0x7f5178102050 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.503+0000 7f517d4ba640 1 -- 172.21.15.78:0/51422153 >> 172.21.15.78:0/51422153 conn(0x7f51780fb3b0 msgr2=0x7f51780fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.503+0000 7f517d4ba640 1 -- 172.21.15.78:0/51422153 shutdown_connections 2024-04-25T01:45:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.503+0000 7f517d4ba640 1 -- 172.21.15.78:0/51422153 wait complete. 2024-04-25T01:45:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f517d4ba640 1 Processor -- start 2024-04-25T01:45:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f517d4ba640 1 -- start start 2024-04-25T01:45:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f517d4ba640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5178100960 0x7f5178197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f517d4ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5178101320 0x7f5178198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f517d4ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5178101c30 0x7f517819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f517d4ba640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f517810c330 con 0x7f5178101c30 2024-04-25T01:45:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f517d4ba640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f517810c1b0 con 0x7f5178100960 2024-04-25T01:45:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f517d4ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f517810c4b0 con 0x7f5178101320 2024-04-25T01:45:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f51777fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5178101c30 0x7f517819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f51777fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5178101c30 0x7f517819f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60150/0 (socket says 172.21.15.78:60150) 2024-04-25T01:45:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f51777fe640 1 -- 172.21.15.78:0/1423047461 learned_addr learned my addr 172.21.15.78:0/1423047461 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f5176ffd640 1 --2- 172.21.15.78:0/1423047461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5178100960 0x7f5178197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f51767fc640 1 --2- 172.21.15.78:0/1423047461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5178101320 0x7f5178198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f51777fe640 1 -- 172.21.15.78:0/1423047461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5178101320 msgr2=0x7f5178198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f51777fe640 1 --2- 172.21.15.78:0/1423047461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5178101320 0x7f5178198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f51777fe640 1 -- 172.21.15.78:0/1423047461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5178100960 msgr2=0x7f5178197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f51777fe640 1 --2- 172.21.15.78:0/1423047461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5178100960 0x7f5178197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.504+0000 7f51777fe640 1 -- 172.21.15.78:0/1423047461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517819f720 con 0x7f5178101c30 2024-04-25T01:45:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.505+0000 7f51777fe640 1 --2- 172.21.15.78:0/1423047461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5178101c30 0x7f517819f0b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f517000eb10 tx=0x7f517000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.505+0000 7f515ffff640 1 -- 172.21.15.78:0/1423047461 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5170004b20 con 0x7f5178101c30 2024-04-25T01:45:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.505+0000 7f517d4ba640 1 -- 172.21.15.78:0/1423047461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517819fa10 con 0x7f5178101c30 2024-04-25T01:45:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.505+0000 7f515ffff640 1 -- 172.21.15.78:0/1423047461 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5170004cc0 con 0x7f5178101c30 2024-04-25T01:45:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.505+0000 7f515ffff640 1 -- 172.21.15.78:0/1423047461 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5170010760 con 0x7f5178101c30 2024-04-25T01:45:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.505+0000 7f517d4ba640 1 -- 172.21.15.78:0/1423047461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f517819ff10 con 0x7f5178101c30 2024-04-25T01:45:32.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.506+0000 7f517d4ba640 1 -- 172.21.15.78:0/1423047461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51781a0140 con 0x7f5178101c30 2024-04-25T01:45:32.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.507+0000 7f515ffff640 1 -- 172.21.15.78:0/1423047461 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5170005020 con 0x7f5178101c30 2024-04-25T01:45:32.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.507+0000 7f515ffff640 1 --2- 172.21.15.78:0/1423047461 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f51600a2b90 0x7f51600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.508+0000 7f5176ffd640 1 --2- 172.21.15.78:0/1423047461 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f51600a2b90 0x7f51600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.508+0000 7f515ffff640 1 -- 172.21.15.78:0/1423047461 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f51700cf9d0 con 0x7f5178101c30 2024-04-25T01:45:32.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.508+0000 7f5176ffd640 1 --2- 172.21.15.78:0/1423047461 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f51600a2b90 0x7f51600a5050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5168005ab0 tx=0x7f5168000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.512+0000 7f515ffff640 1 -- 172.21.15.78:0/1423047461 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f517009c550 con 0x7f5178101c30 2024-04-25T01:45:32.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.688+0000 7f517d4ba640 1 -- 172.21.15.78:0/1423047461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51781a03b0 con 0x7f5178101c30 2024-04-25T01:45:32.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.689+0000 7f515ffff640 1 -- 172.21.15.78:0/1423047461 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24844 (secure 0 0 0) 0x7f517009be50 con 0x7f5178101c30 2024-04-25T01:45:32.682 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:45:32.682 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":94,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:45:18.989934+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:45:32.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.692+0000 7f517d4ba640 1 -- 172.21.15.78:0/1423047461 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f51600a2b90 msgr2=0x7f51600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.692+0000 7f517d4ba640 1 --2- 172.21.15.78:0/1423047461 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f51600a2b90 0x7f51600a5050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5168005ab0 tx=0x7f5168000bc0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.692+0000 7f517d4ba640 1 -- 172.21.15.78:0/1423047461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5178101c30 msgr2=0x7f517819f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.692+0000 7f517d4ba640 1 --2- 172.21.15.78:0/1423047461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5178101c30 0x7f517819f0b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f517000eb10 tx=0x7f517000c2d0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.692+0000 7f517d4ba640 1 -- 172.21.15.78:0/1423047461 shutdown_connections 2024-04-25T01:45:32.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.692+0000 7f517d4ba640 1 --2- 172.21.15.78:0/1423047461 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f51600a2b90 0x7f51600a5050 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.692+0000 7f517d4ba640 1 --2- 172.21.15.78:0/1423047461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5178101c30 0x7f517819f0b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.692+0000 7f517d4ba640 1 --2- 172.21.15.78:0/1423047461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5178101320 0x7f5178198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.692+0000 7f517d4ba640 1 --2- 172.21.15.78:0/1423047461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5178100960 0x7f5178197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.692+0000 7f517d4ba640 1 -- 172.21.15.78:0/1423047461 >> 172.21.15.78:0/1423047461 conn(0x7f51780fb3b0 msgr2=0x7f51780fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.692+0000 7f517d4ba640 1 -- 172.21.15.78:0/1423047461 shutdown_connections 2024-04-25T01:45:32.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.692+0000 7f517d4ba640 1 -- 172.21.15.78:0/1423047461 wait complete. 2024-04-25T01:45:32.697 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T01:43:17.355192+0000', 'modified': '2024-04-25T01:43:44.435289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24491, 'mds_1': 24455, 'mds_2': 24448, 'mds_3': 24392, 'mds_4': 24379}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14634': {'gid': 14634, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6839/1337519666', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 1337519666}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 1337519666}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14646': {'gid': 14646, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6841/1963410683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6840', 'nonce': 1963410683}, {'type': 'v1', 'addr': '172.21.15.78:6841', 'nonce': 1963410683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24379': {'gid': 24379, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.196:6833/4110091360', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 4110091360}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 4110091360}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24392': {'gid': 24392, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.152:6833/923579368', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 923579368}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 923579368}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/2927651724', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 2927651724}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 2927651724}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6835/4251210610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6834', 'nonce': 4251210610}, {'type': 'v1', 'addr': '172.21.15.152:6835', 'nonce': 4251210610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6837/1117133390', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 1117133390}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 1117133390}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.196:6839/460829167', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 460829167}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 460829167}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.152:6837/4112159743', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6836', 'nonce': 4112159743}, {'type': 'v1', 'addr': '172.21.15.152:6837', 'nonce': 4112159743}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6839/1969344843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 1969344843}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 1969344843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24491, 'qdb_cluster': [24379, 24491, 24455, 24392, 24448]} 2024-04-25T01:45:32.697 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T01:45:32.697 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-25T01:45:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.804+0000 7f9802aee640 1 -- 172.21.15.78:0/3619175205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97fc101280 msgr2=0x7f97fc108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.804+0000 7f9802aee640 1 --2- 172.21.15.78:0/3619175205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97fc101280 0x7f97fc108b20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009ca0 tx=0x7f97f402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.804+0000 7f9802aee640 1 -- 172.21.15.78:0/3619175205 shutdown_connections 2024-04-25T01:45:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.804+0000 7f9802aee640 1 --2- 172.21.15.78:0/3619175205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97fc101280 0x7f97fc108b20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.804+0000 7f9802aee640 1 --2- 172.21.15.78:0/3619175205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97fc1008c0 0x7f97fc100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.804+0000 7f9802aee640 1 --2- 172.21.15.78:0/3619175205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97fc101c70 0x7f97fc102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.804+0000 7f9802aee640 1 -- 172.21.15.78:0/3619175205 >> 172.21.15.78:0/3619175205 conn(0x7f97fc0fb370 msgr2=0x7f97fc0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.804+0000 7f9802aee640 1 -- 172.21.15.78:0/3619175205 shutdown_connections 2024-04-25T01:45:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.804+0000 7f9802aee640 1 -- 172.21.15.78:0/3619175205 wait complete. 2024-04-25T01:45:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f9802aee640 1 Processor -- start 2024-04-25T01:45:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f9802aee640 1 -- start start 2024-04-25T01:45:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f9802aee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97fc1008c0 0x7f97fc100340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f9802aee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97fc101280 0x7f97fc19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f9802aee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97fc101c70 0x7f97fc19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f9802aee640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc10c310 con 0x7f97fc101c70 2024-04-25T01:45:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f9802aee640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc10c190 con 0x7f97fc1008c0 2024-04-25T01:45:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f9802aee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc10c490 con 0x7f97fc101280 2024-04-25T01:45:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f980108b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97fc101c70 0x7f97fc19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f980108b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97fc101c70 0x7f97fc19ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60158/0 (socket says 172.21.15.78:60158) 2024-04-25T01:45:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f980108b640 1 -- 172.21.15.78:0/688630363 learned_addr learned my addr 172.21.15.78:0/688630363 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f980108b640 1 -- 172.21.15.78:0/688630363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97fc101280 msgr2=0x7f97fc19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f980108b640 1 --2- 172.21.15.78:0/688630363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97fc101280 0x7f97fc19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f980088a640 1 --2- 172.21.15.78:0/688630363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97fc1008c0 0x7f97fc100340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f980108b640 1 -- 172.21.15.78:0/688630363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97fc1008c0 msgr2=0x7f97fc100340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f980108b640 1 --2- 172.21.15.78:0/688630363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97fc1008c0 0x7f97fc100340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f980108b640 1 -- 172.21.15.78:0/688630363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc19f530 con 0x7f97fc101c70 2024-04-25T01:45:32.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f980108b640 1 --2- 172.21.15.78:0/688630363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97fc101c70 0x7f97fc19ec70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f97f40098d0 tx=0x7f97f402f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f97f9ffb640 1 -- 172.21.15.78:0/688630363 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f402fc30 con 0x7f97fc101c70 2024-04-25T01:45:32.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f97f9ffb640 1 -- 172.21.15.78:0/688630363 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f4003e50 con 0x7f97fc101c70 2024-04-25T01:45:32.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f9802aee640 1 -- 172.21.15.78:0/688630363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97fc19f7c0 con 0x7f97fc101c70 2024-04-25T01:45:32.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.805+0000 7f97f9ffb640 1 -- 172.21.15.78:0/688630363 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f40305c0 con 0x7f97fc101c70 2024-04-25T01:45:32.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.806+0000 7f9802aee640 1 -- 172.21.15.78:0/688630363 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97fc19fd60 con 0x7f97fc101c70 2024-04-25T01:45:32.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.808+0000 7f97f9ffb640 1 -- 172.21.15.78:0/688630363 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f97f4003970 con 0x7f97fc101c70 2024-04-25T01:45:32.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.808+0000 7f98057fa740 1 Processor -- start 2024-04-25T01:45:32.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.808+0000 7f98057fa740 1 -- start start 2024-04-25T01:45:32.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.808+0000 7f98057fa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3ab860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.808+0000 7f98057fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 0x5609ca3b0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.808+0000 7f98057fa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 0x5609ca3b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.808+0000 7f98057fa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5609ca30ef00 con 0x5609ca3ab440 2024-04-25T01:45:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.808+0000 7f98057fa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5609ca2b7750 con 0x5609ca3b0fa0 2024-04-25T01:45:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.808+0000 7f98057fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5609ca30aec0 con 0x5609ca3abe30 2024-04-25T01:45:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.808+0000 7f97f9ffb640 1 --2- 172.21.15.78:0/688630363 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f97ec0a2c10 0x7f97ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.808+0000 7f97fbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.808+0000 7f97fbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3ab860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60172/0 (socket says 172.21.15.78:60172) 2024-04-25T01:45:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.808+0000 7f97fbfff640 1 -- 172.21.15.78:0/4072254259 learned_addr learned my addr 172.21.15.78:0/4072254259 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.808+0000 7f980108b640 1 --2- 172.21.15.78:0/688630363 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f97ec0a2c10 0x7f97ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.809+0000 7f97f9ffb640 1 -- 172.21.15.78:0/688630363 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f97f40f0880 con 0x7f97fc101c70 2024-04-25T01:45:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.809+0000 7f980088a640 1 --2- 172.21.15.78:0/4072254259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 0x5609ca3b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.809+0000 7f97fbfff640 1 -- 172.21.15.78:0/4072254259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 msgr2=0x5609ca3b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.809+0000 7f97fbfff640 1 --2- 172.21.15.78:0/4072254259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 0x5609ca3b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.809+0000 7f97fbfff640 1 -- 172.21.15.78:0/4072254259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 msgr2=0x5609ca3b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.809+0000 7f97fbfff640 1 --2- 172.21.15.78:0/4072254259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 0x5609ca3b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.809+0000 7f97fbfff640 1 -- 172.21.15.78:0/4072254259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609ca0232d0 con 0x5609ca3ab440 2024-04-25T01:45:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.809+0000 7f97fbfff640 1 --2- 172.21.15.78:0/4072254259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3ab860 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f97e800ab50 tx=0x7f97e800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b2c93ae4a7b0e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.809+0000 7f980108b640 1 --2- 172.21.15.78:0/688630363 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f97ec0a2c10 0x7f97ec0a50d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f97f40b9160 tx=0x7f97f40b21e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.809+0000 7f97e6ffd640 1 -- 172.21.15.78:0/4072254259 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8005220 con 0x5609ca3ab440 2024-04-25T01:45:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.809+0000 7f97e6ffd640 1 -- 172.21.15.78:0/4072254259 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97e800ae50 con 0x5609ca3ab440 2024-04-25T01:45:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.809+0000 7f97e6ffd640 1 -- 172.21.15.78:0/4072254259 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e80106a0 con 0x5609ca3ab440 2024-04-25T01:45:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 -- 172.21.15.78:0/4072254259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 msgr2=0x5609ca3ab860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 --2- 172.21.15.78:0/4072254259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3ab860 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f97e800ab50 tx=0x7f97e800e6c0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 -- 172.21.15.78:0/4072254259 shutdown_connections 2024-04-25T01:45:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 --2- 172.21.15.78:0/4072254259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 0x5609ca3b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 --2- 172.21.15.78:0/4072254259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 0x5609ca3b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 --2- 172.21.15.78:0/4072254259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3ab860 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 -- 172.21.15.78:0/4072254259 >> 172.21.15.78:0/4072254259 conn(0x5609ca3a6ab0 msgr2=0x5609ca3a6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 -- 172.21.15.78:0/4072254259 shutdown_connections 2024-04-25T01:45:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 -- 172.21.15.78:0/4072254259 wait complete. 2024-04-25T01:45:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 -- start start 2024-04-25T01:45:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 Processor -- start 2024-04-25T01:45:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3ab440 0x5609ca3b5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3abe30 0x5609ca3b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 0x5609ca3bcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.810+0000 7f98057fa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5609ca3b3f50 con 0x5609ca3abe30 2024-04-25T01:45:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f98057fa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5609ca2cacf0 con 0x5609ca3b0fa0 2024-04-25T01:45:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f98057fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5609ca3b40d0 con 0x5609ca3ab440 2024-04-25T01:45:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f98057fa740 10 client.0 _ll_register_callbacks cb 0x5609ca2ca6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f980088a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3abe30 0x5609ca3b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f980088a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3abe30 0x5609ca3b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60176/0 (socket says 172.21.15.78:60176) 2024-04-25T01:45:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f980088a640 1 -- 172.21.15.78:0/257983108 learned_addr learned my addr 172.21.15.78:0/257983108 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:32.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f97fbfff640 1 --2- 172.21.15.78:0/257983108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 0x5609ca3bcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f980088a640 1 -- 172.21.15.78:0/257983108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3ab440 msgr2=0x5609ca3b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f980088a640 1 --2- 172.21.15.78:0/257983108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3ab440 0x5609ca3b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f980088a640 1 -- 172.21.15.78:0/257983108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 msgr2=0x5609ca3bcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f980088a640 1 --2- 172.21.15.78:0/257983108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 0x5609ca3bcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f980088a640 1 -- 172.21.15.78:0/257983108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609ca0ed6b0 con 0x5609ca3abe30 2024-04-25T01:45:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f980088a640 1 --2- 172.21.15.78:0/257983108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3abe30 0x5609ca3b5d90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f97f000f6d0 tx=0x7f97f000fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be3a3936c841e2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f97dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f97dbfff640 1 -- 172.21.15.78:0/257983108 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0016070 con 0x5609ca3abe30 2024-04-25T01:45:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f97dbfff640 1 -- 172.21.15.78:0/257983108 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f97f000baa0 con 0x5609ca3abe30 2024-04-25T01:45:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f98057fa740 1 -- 172.21.15.78:0/257983108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97e800ae50 con 0x5609ca3abe30 2024-04-25T01:45:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f97dbfff640 1 -- 172.21.15.78:0/257983108 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f97f0025050 con 0x5609ca3abe30 2024-04-25T01:45:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f98057fa740 10 client.15450 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.811+0000 7f98057fa740 1 -- 172.21.15.78:0/257983108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5609ca3bfe80 con 0x5609ca3abe30 2024-04-25T01:45:32.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.812+0000 7f97dbfff640 1 -- 172.21.15.78:0/257983108 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97e800ae50 con 0x5609ca3abe30 2024-04-25T01:45:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.812+0000 7f97dbfff640 1 -- 172.21.15.78:0/257983108 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f97f001a490 con 0x5609ca3abe30 2024-04-25T01:45:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.812+0000 7f98057fa740 10 client.15450 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.812+0000 7f98057fa740 10 client.15450 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-25T01:45:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.812+0000 7f98057fa740 20 client.15450 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.812+0000 7f98057fa740 1 --2- 172.21.15.78:0/257983108 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x5609ca3c25a0 0x5609ca3c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.812+0000 7f98057fa740 4 client.15450 mds_command: new command op to 24379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:45:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.812+0000 7f98057fa740 1 -- 172.21.15.78:0/257983108 --> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609ca31c440 con 0x5609ca3c25a0 2024-04-25T01:45:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.812+0000 7f97fbfff640 1 --2- 172.21.15.78:0/257983108 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x5609ca3c25a0 0x5609ca3c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.813+0000 7f97dbfff640 10 client.15450 ms_handle_connect on v2:172.21.15.196:6832/4110091360 2024-04-25T01:45:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.813+0000 7f97fbfff640 1 --2- 172.21.15.78:0/257983108 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x5609ca3c25a0 0x5609ca3c3a10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7069f934b929a23a server_cookie=85ab7ae0e374bb0c in_seq=0 out_seq=0 2024-04-25T01:45:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.813+0000 7f97dbfff640 1 -- 172.21.15.78:0/257983108 <== mds.4 v2:172.21.15.196:6832/4110091360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5609ca31c440 con 0x5609ca3c25a0 2024-04-25T01:45:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.813+0000 7f97dbfff640 10 client.15450 handle_command_reply: tid=0 2024-04-25T01:45:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.813+0000 7f98057fa740 1 client.15450 shutdown 2024-04-25T01:45:32.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.813+0000 7f98057fa740 1 -- 172.21.15.78:0/257983108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3abe30 msgr2=0x5609ca3b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.813+0000 7f98057fa740 1 --2- 172.21.15.78:0/257983108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3abe30 0x5609ca3b5d90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f97f000f6d0 tx=0x7f97f000fb90 comp rx=0 tx=0).stop 2024-04-25T01:45:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.813+0000 7f98057fa740 1 -- 172.21.15.78:0/257983108 shutdown_connections 2024-04-25T01:45:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.813+0000 7f98057fa740 1 --2- 172.21.15.78:0/257983108 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x5609ca3c25a0 0x5609ca3c3a10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.813+0000 7f98057fa740 1 --2- 172.21.15.78:0/257983108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 0x5609ca3bcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.813+0000 7f98057fa740 1 --2- 172.21.15.78:0/257983108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3abe30 0x5609ca3b5d90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.813+0000 7f97dbfff640 0 client.15450 ms_handle_reset on v2:172.21.15.196:6832/4110091360 2024-04-25T01:45:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.813+0000 7f98057fa740 1 --2- 172.21.15.78:0/257983108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3ab440 0x5609ca3b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.813+0000 7f98057fa740 1 -- 172.21.15.78:0/257983108 >> 172.21.15.78:0/257983108 conn(0x5609ca3a6ab0 msgr2=0x5609ca3a5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.814+0000 7f98057fa740 1 -- 172.21.15.78:0/257983108 shutdown_connections 2024-04-25T01:45:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.814+0000 7f98057fa740 1 -- 172.21.15.78:0/257983108 wait complete. 2024-04-25T01:45:32.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.814+0000 7f98057fa740 20 client.15450 trim_cache size 0 max 16384 2024-04-25T01:45:32.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.831+0000 7f98057fa740 1 Processor -- start 2024-04-25T01:45:32.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.831+0000 7f98057fa740 1 -- start start 2024-04-25T01:45:32.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.831+0000 7f98057fa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.831+0000 7f98057fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 0x5609ca3bcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.831+0000 7f98057fa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 0x5609ca3ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.831+0000 7f98057fa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5609ca3c1d50 con 0x5609ca3ab440 2024-04-25T01:45:32.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.831+0000 7f98057fa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5609ca3c2420 con 0x5609ca3b0fa0 2024-04-25T01:45:32.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.831+0000 7f98057fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5609ca3c2af0 con 0x5609ca3abe30 2024-04-25T01:45:32.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.832+0000 7f97fbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.832+0000 7f97fbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60192/0 (socket says 172.21.15.78:60192) 2024-04-25T01:45:32.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.832+0000 7f97fbfff640 1 -- 172.21.15.78:0/1424188497 learned_addr learned my addr 172.21.15.78:0/1424188497 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:32.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.832+0000 7f97fbfff640 1 -- 172.21.15.78:0/1424188497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 msgr2=0x5609ca3bcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.832+0000 7f97fbfff640 1 --2- 172.21.15.78:0/1424188497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 0x5609ca3bcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.832+0000 7f97fbfff640 1 -- 172.21.15.78:0/1424188497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 msgr2=0x5609ca3ac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.832+0000 7f97fbfff640 1 --2- 172.21.15.78:0/1424188497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 0x5609ca3ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.832+0000 7f97fbfff640 1 -- 172.21.15.78:0/1424188497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609ca3a4fc0 con 0x5609ca3ab440 2024-04-25T01:45:32.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.832+0000 7f97fbfff640 1 --2- 172.21.15.78:0/1424188497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3b0860 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x5609ca3ac690 tx=0x7f97e800a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84eabc938a5c5105 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.832+0000 7f97e67fc640 1 -- 172.21.15.78:0/1424188497 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8004220 con 0x5609ca3ab440 2024-04-25T01:45:32.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.832+0000 7f97e67fc640 1 -- 172.21.15.78:0/1424188497 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97e8025920 con 0x5609ca3ab440 2024-04-25T01:45:32.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.832+0000 7f97e67fc640 1 -- 172.21.15.78:0/1424188497 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8003c20 con 0x5609ca3ab440 2024-04-25T01:45:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 -- 172.21.15.78:0/1424188497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 msgr2=0x5609ca3b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 --2- 172.21.15.78:0/1424188497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3b0860 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x5609ca3ac690 tx=0x7f97e800a510 comp rx=0 tx=0).stop 2024-04-25T01:45:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 -- 172.21.15.78:0/1424188497 shutdown_connections 2024-04-25T01:45:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 --2- 172.21.15.78:0/1424188497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 0x5609ca3ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 --2- 172.21.15.78:0/1424188497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 0x5609ca3bcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 --2- 172.21.15.78:0/1424188497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3b0860 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 -- 172.21.15.78:0/1424188497 >> 172.21.15.78:0/1424188497 conn(0x5609ca3a6ab0 msgr2=0x5609ca3a1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 -- 172.21.15.78:0/1424188497 shutdown_connections 2024-04-25T01:45:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 -- 172.21.15.78:0/1424188497 wait complete. 2024-04-25T01:45:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 -- start start 2024-04-25T01:45:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 Processor -- start 2024-04-25T01:45:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3bad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 0x5609ca424530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 0x5609ca426a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5609ca3c3890 con 0x5609ca3ab440 2024-04-25T01:45:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5609ca3c31c0 con 0x5609ca3b0fa0 2024-04-25T01:45:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5609ca3c3f60 con 0x5609ca3abe30 2024-04-25T01:45:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.833+0000 7f98057fa740 10 client.0 _ll_register_callbacks cb 0x7f97f000dea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f97fbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3bad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f97fbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3bad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60200/0 (socket says 172.21.15.78:60200) 2024-04-25T01:45:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f97fbfff640 1 -- 172.21.15.78:0/685639840 learned_addr learned my addr 172.21.15.78:0/685639840 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f980088a640 1 --2- 172.21.15.78:0/685639840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 0x5609ca424530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f97fbfff640 1 -- 172.21.15.78:0/685639840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 msgr2=0x5609ca424530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f97fbfff640 1 --2- 172.21.15.78:0/685639840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 0x5609ca424530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f97fbfff640 1 -- 172.21.15.78:0/685639840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 msgr2=0x5609ca426a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f97fbfff640 1 --2- 172.21.15.78:0/685639840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 0x5609ca426a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f97fbfff640 1 -- 172.21.15.78:0/685639840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609ca427120 con 0x5609ca3ab440 2024-04-25T01:45:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f980088a640 1 --2- 172.21.15.78:0/685639840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 0x5609ca424530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f97fbfff640 1 --2- 172.21.15.78:0/685639840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3bad80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f97e80057c0 tx=0x7f97e80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d6f9d8d000bc75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f97dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f97dbfff640 1 -- 172.21.15.78:0/685639840 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8010590 con 0x5609ca3ab440 2024-04-25T01:45:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f97dbfff640 1 -- 172.21.15.78:0/685639840 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f97e8010730 con 0x5609ca3ab440 2024-04-25T01:45:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f98057fa740 1 -- 172.21.15.78:0/685639840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5609ca316790 con 0x5609ca3ab440 2024-04-25T01:45:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f97dbfff640 1 -- 172.21.15.78:0/685639840 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f97e8010930 con 0x5609ca3ab440 2024-04-25T01:45:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f98057fa740 10 client.15456 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f98057fa740 1 -- 172.21.15.78:0/685639840 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5609ca428420 con 0x5609ca3ab440 2024-04-25T01:45:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.834+0000 7f97dbfff640 1 -- 172.21.15.78:0/685639840 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5609ca316790 con 0x5609ca3ab440 2024-04-25T01:45:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.835+0000 7f97dbfff640 1 -- 172.21.15.78:0/685639840 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f97e800f040 con 0x5609ca3ab440 2024-04-25T01:45:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.835+0000 7f98057fa740 10 client.15456 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:32.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.835+0000 7f98057fa740 10 client.15456 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-25T01:45:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.835+0000 7f98057fa740 20 client.15456 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.835+0000 7f98057fa740 1 --2- 172.21.15.78:0/685639840 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x5609ca42c890 0x5609ca42b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.835+0000 7f98057fa740 4 client.15456 mds_command: new command op to 24379 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:45:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.835+0000 7f98057fa740 1 -- 172.21.15.78:0/685639840 --> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] -- command(tid 0: {"prefix": "status"}) -- 0x5609ca294b80 con 0x5609ca42c890 2024-04-25T01:45:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.835+0000 7f980088a640 1 --2- 172.21.15.78:0/685639840 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x5609ca42c890 0x5609ca42b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.835+0000 7f97dbfff640 10 client.15456 ms_handle_connect on v2:172.21.15.196:6832/4110091360 2024-04-25T01:45:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.835+0000 7f980088a640 1 --2- 172.21.15.78:0/685639840 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x5609ca42c890 0x5609ca42b5a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=83960bfbd18400 server_cookie=12c86858fb5409a5 in_seq=0 out_seq=0 2024-04-25T01:45:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f97dbfff640 1 -- 172.21.15.78:0/685639840 <== mds.4 v2:172.21.15.196:6832/4110091360 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5609ca294b80 con 0x5609ca42c890 2024-04-25T01:45:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f97dbfff640 10 client.15456 handle_command_reply: tid=0 2024-04-25T01:45:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f98057fa740 1 client.15456 shutdown 2024-04-25T01:45:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f98057fa740 1 -- 172.21.15.78:0/685639840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 msgr2=0x5609ca3bad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f98057fa740 1 --2- 172.21.15.78:0/685639840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3bad80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f97e80057c0 tx=0x7f97e80035f0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f98057fa740 1 -- 172.21.15.78:0/685639840 shutdown_connections 2024-04-25T01:45:32.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f98057fa740 1 --2- 172.21.15.78:0/685639840 >> [v2:172.21.15.196:6832/4110091360,v1:172.21.15.196:6833/4110091360] conn(0x5609ca42c890 0x5609ca42b5a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f98057fa740 1 --2- 172.21.15.78:0/685639840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609ca3b0fa0 0x5609ca426a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f97dbfff640 0 client.15456 ms_handle_reset on v2:172.21.15.196:6832/4110091360 2024-04-25T01:45:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f98057fa740 1 --2- 172.21.15.78:0/685639840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609ca3abe30 0x5609ca424530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f98057fa740 1 --2- 172.21.15.78:0/685639840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5609ca3ab440 0x5609ca3bad80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f98057fa740 1 -- 172.21.15.78:0/685639840 >> 172.21.15.78:0/685639840 conn(0x5609ca3a6ab0 msgr2=0x5609ca3ac6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f98057fa740 1 -- 172.21.15.78:0/685639840 shutdown_connections 2024-04-25T01:45:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f98057fa740 1 -- 172.21.15.78:0/685639840 wait complete. 2024-04-25T01:45:32.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.836+0000 7f98057fa740 20 client.15456 trim_cache size 0 max 16384 2024-04-25T01:45:32.835 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:45:32.835 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "df36bea8-02a3-11ef-bc93-c7b262605968", 2024-04-25T01:45:32.835 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 4, 2024-04-25T01:45:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24379, 2024-04-25T01:45:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T01:45:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T01:45:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T01:45:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 109.40358899500001, 2024-04-25T01:45:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-04-25T01:45:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 94, 2024-04-25T01:45:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T01:45:32.836 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 172.70606313799999 2024-04-25T01:45:32.836 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:45:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.837+0000 7f9802aee640 1 -- 172.21.15.78:0/688630363 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f97ec0a2c10 msgr2=0x7f97ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.837+0000 7f9802aee640 1 --2- 172.21.15.78:0/688630363 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f97ec0a2c10 0x7f97ec0a50d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f97f40b9160 tx=0x7f97f40b21e0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.837+0000 7f9802aee640 1 -- 172.21.15.78:0/688630363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97fc101c70 msgr2=0x7f97fc19ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.837+0000 7f9802aee640 1 --2- 172.21.15.78:0/688630363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97fc101c70 0x7f97fc19ec70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f97f40098d0 tx=0x7f97f402f840 comp rx=0 tx=0).stop 2024-04-25T01:45:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.837+0000 7f9802aee640 1 -- 172.21.15.78:0/688630363 shutdown_connections 2024-04-25T01:45:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.837+0000 7f9802aee640 1 --2- 172.21.15.78:0/688630363 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f97ec0a2c10 0x7f97ec0a50d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.837+0000 7f9802aee640 1 --2- 172.21.15.78:0/688630363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97fc101c70 0x7f97fc19ec70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.837+0000 7f9802aee640 1 --2- 172.21.15.78:0/688630363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97fc101280 0x7f97fc19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.837+0000 7f9802aee640 1 --2- 172.21.15.78:0/688630363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97fc1008c0 0x7f97fc100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.837+0000 7f9802aee640 1 -- 172.21.15.78:0/688630363 >> 172.21.15.78:0/688630363 conn(0x7f97fc0fb370 msgr2=0x7f97fc104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.837+0000 7f9802aee640 1 -- 172.21.15.78:0/688630363 shutdown_connections 2024-04-25T01:45:32.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.837+0000 7f9802aee640 1 -- 172.21.15.78:0/688630363 wait complete. 2024-04-25T01:45:32.840 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-25T01:45:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.946+0000 7fe432ff9640 1 -- 172.21.15.78:0/2790623668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c102980 msgr2=0x7fe42c102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.946+0000 7fe432ff9640 1 --2- 172.21.15.78:0/2790623668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c102980 0x7fe42c102e00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009900 tx=0x7fe41c02a410 comp rx=0 tx=0).stop 2024-04-25T01:45:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.946+0000 7fe432ff9640 1 -- 172.21.15.78:0/2790623668 shutdown_connections 2024-04-25T01:45:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.946+0000 7fe432ff9640 1 --2- 172.21.15.78:0/2790623668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe42c103340 0x7fe42c109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.946+0000 7fe432ff9640 1 --2- 172.21.15.78:0/2790623668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c102980 0x7fe42c102e00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.946+0000 7fe432ff9640 1 --2- 172.21.15.78:0/2790623668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe42c101790 0x7fe42c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.946+0000 7fe432ff9640 1 -- 172.21.15.78:0/2790623668 >> 172.21.15.78:0/2790623668 conn(0x7fe42c0fcd20 msgr2=0x7fe42c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.946+0000 7fe432ff9640 1 -- 172.21.15.78:0/2790623668 shutdown_connections 2024-04-25T01:45:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.946+0000 7fe432ff9640 1 -- 172.21.15.78:0/2790623668 wait complete. 2024-04-25T01:45:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe432ff9640 1 Processor -- start 2024-04-25T01:45:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe432ff9640 1 -- start start 2024-04-25T01:45:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe432ff9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c101790 0x7fe42c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe432ff9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe42c102980 0x7fe42c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe432ff9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe42c103340 0x7fe42c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe432ff9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c10d410 con 0x7fe42c101790 2024-04-25T01:45:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe432ff9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c10d290 con 0x7fe42c103340 2024-04-25T01:45:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe432ff9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c10d590 con 0x7fe42c102980 2024-04-25T01:45:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe430d95640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c101790 0x7fe42c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe430d95640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c101790 0x7fe42c197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60230/0 (socket says 172.21.15.78:60230) 2024-04-25T01:45:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe430d95640 1 -- 172.21.15.78:0/2589330404 learned_addr learned my addr 172.21.15.78:0/2589330404 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe42bfff640 1 --2- 172.21.15.78:0/2589330404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe42c102980 0x7fe42c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe431596640 1 --2- 172.21.15.78:0/2589330404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe42c103340 0x7fe42c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe430d95640 1 -- 172.21.15.78:0/2589330404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe42c102980 msgr2=0x7fe42c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe430d95640 1 --2- 172.21.15.78:0/2589330404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe42c102980 0x7fe42c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe430d95640 1 -- 172.21.15.78:0/2589330404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe42c103340 msgr2=0x7fe42c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe430d95640 1 --2- 172.21.15.78:0/2589330404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe42c103340 0x7fe42c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe430d95640 1 -- 172.21.15.78:0/2589330404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42c19f6e0 con 0x7fe42c101790 2024-04-25T01:45:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe430d95640 1 --2- 172.21.15.78:0/2589330404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c101790 0x7fe42c197ac0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe42400ea80 tx=0x7fe42400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.947+0000 7fe429ffb640 1 -- 172.21.15.78:0/2589330404 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42400ce40 con 0x7fe42c101790 2024-04-25T01:45:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.948+0000 7fe429ffb640 1 -- 172.21.15.78:0/2589330404 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe424011950 con 0x7fe42c101790 2024-04-25T01:45:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.948+0000 7fe432ff9640 1 -- 172.21.15.78:0/2589330404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42c19f9d0 con 0x7fe42c101790 2024-04-25T01:45:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.948+0000 7fe429ffb640 1 -- 172.21.15.78:0/2589330404 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42401ab80 con 0x7fe42c101790 2024-04-25T01:45:32.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.948+0000 7fe432ff9640 1 -- 172.21.15.78:0/2589330404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe42c19fef0 con 0x7fe42c101790 2024-04-25T01:45:32.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.950+0000 7fe429ffb640 1 -- 172.21.15.78:0/2589330404 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe424011b00 con 0x7fe42c101790 2024-04-25T01:45:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe435de5740 1 Processor -- start 2024-04-25T01:45:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe435de5740 1 -- start start 2024-04-25T01:45:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe429ffb640 1 --2- 172.21.15.78:0/2589330404 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4180a2b90 0x7fe4180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe435de5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9c850 0x5654f3f9cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe435de5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3f9d240 0x5654f3fa1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe435de5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3fa23b0 0x5654f3fa4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe435de5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5654f3efbf00 con 0x5654f3fa23b0 2024-04-25T01:45:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe435de5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5654f3ea4750 con 0x5654f3f9d240 2024-04-25T01:45:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe435de5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654f3ef7ec0 con 0x5654f3f9c850 2024-04-25T01:45:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe429ffb640 1 -- 172.21.15.78:0/2589330404 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fe4240cfa20 con 0x7fe42c101790 2024-04-25T01:45:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe431596640 1 --2- 172.21.15.78:0/2589330404 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4180a2b90 0x7fe4180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe430d95640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9c850 0x5654f3f9cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe431596640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3fa23b0 0x5654f3fa4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe430d95640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9c850 0x5654f3f9cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.78:38994/0 (socket says 172.21.15.78:38994) 2024-04-25T01:45:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe430d95640 1 -- 172.21.15.78:0/1273059739 learned_addr learned my addr 172.21.15.78:0/1273059739 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe42bfff640 1 --2- 172.21.15.78:0/1273059739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3f9d240 0x5654f3fa1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe431596640 1 -- 172.21.15.78:0/1273059739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9c850 msgr2=0x5654f3f9cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe431596640 1 --2- 172.21.15.78:0/1273059739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9c850 0x5654f3f9cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe431596640 1 -- 172.21.15.78:0/1273059739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3f9d240 msgr2=0x5654f3fa1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.951+0000 7fe431596640 1 --2- 172.21.15.78:0/1273059739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3f9d240 0x5654f3fa1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.952+0000 7fe431596640 1 -- 172.21.15.78:0/1273059739 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f3c102d0 con 0x5654f3fa23b0 2024-04-25T01:45:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.952+0000 7fe430d95640 1 --2- 172.21.15.78:0/1273059739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9c850 0x5654f3f9cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:45:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.952+0000 7fe42bfff640 1 --2- 172.21.15.78:0/1273059739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3f9d240 0x5654f3fa1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:45:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.952+0000 7fe431596640 1 --2- 172.21.15.78:0/2589330404 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4180a2b90 0x7fe4180a5050 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe41400b590 tx=0x7fe41400baa0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.952+0000 7fe431596640 1 --2- 172.21.15.78:0/1273059739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3fa23b0 0x5654f3fa4840 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe414000b60 tx=0x7fe41401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fee078b88ac3b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.952+0000 7fe422ffd640 1 -- 172.21.15.78:0/1273059739 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414024070 con 0x5654f3fa23b0 2024-04-25T01:45:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.952+0000 7fe422ffd640 1 -- 172.21.15.78:0/1273059739 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe414012e50 con 0x5654f3fa23b0 2024-04-25T01:45:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.952+0000 7fe422ffd640 1 -- 172.21.15.78:0/1273059739 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414014e60 con 0x5654f3fa23b0 2024-04-25T01:45:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.952+0000 7fe435de5740 1 -- 172.21.15.78:0/1273059739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3fa23b0 msgr2=0x5654f3fa4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.952+0000 7fe435de5740 1 --2- 172.21.15.78:0/1273059739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3fa23b0 0x5654f3fa4840 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe414000b60 tx=0x7fe41401c000 comp rx=0 tx=0).stop 2024-04-25T01:45:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 -- 172.21.15.78:0/1273059739 shutdown_connections 2024-04-25T01:45:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 --2- 172.21.15.78:0/1273059739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3fa23b0 0x5654f3fa4840 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 --2- 172.21.15.78:0/1273059739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3f9d240 0x5654f3fa1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 --2- 172.21.15.78:0/1273059739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9c850 0x5654f3f9cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 -- 172.21.15.78:0/1273059739 >> 172.21.15.78:0/1273059739 conn(0x5654f3f97ec0 msgr2=0x5654f3f982c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 -- 172.21.15.78:0/1273059739 shutdown_connections 2024-04-25T01:45:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 -- 172.21.15.78:0/1273059739 wait complete. 2024-04-25T01:45:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 -- start start 2024-04-25T01:45:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 Processor -- start 2024-04-25T01:45:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9c850 0x5654f3fa6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3f9d240 0x5654f3fa7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3fa23b0 0x5654f3fae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5654f3fa5300 con 0x5654f3f9d240 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5654f3eb7cf0 con 0x5654f3fa23b0 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654f3fa5480 con 0x5654f3f9c850 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe435de5740 10 client.0 _ll_register_callbacks cb 0x5654f3eb76e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe42bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3f9d240 0x5654f3fa7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe42bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3f9d240 0x5654f3fa7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60258/0 (socket says 172.21.15.78:60258) 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.953+0000 7fe42bfff640 1 -- 172.21.15.78:0/2086028370 learned_addr learned my addr 172.21.15.78:0/2086028370 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe42bfff640 1 -- 172.21.15.78:0/2086028370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9c850 msgr2=0x5654f3fa6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe42bfff640 1 --2- 172.21.15.78:0/2086028370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9c850 0x5654f3fa6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe431596640 1 --2- 172.21.15.78:0/2086028370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3fa23b0 0x5654f3fae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe42bfff640 1 -- 172.21.15.78:0/2086028370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3fa23b0 msgr2=0x5654f3fae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe42bfff640 1 --2- 172.21.15.78:0/2086028370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3fa23b0 0x5654f3fae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe42bfff640 1 -- 172.21.15.78:0/2086028370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f3cda6b0 con 0x5654f3f9d240 2024-04-25T01:45:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe42bfff640 1 --2- 172.21.15.78:0/2086028370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3f9d240 0x5654f3fa7100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe41c031610 tx=0x7fe41c0318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fe254450899216 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe40bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe40bfff640 1 -- 172.21.15.78:0/2086028370 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c035040 con 0x5654f3f9d240 2024-04-25T01:45:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe40bfff640 1 -- 172.21.15.78:0/2086028370 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fe41c004450 con 0x5654f3f9d240 2024-04-25T01:45:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe435de5740 1 -- 172.21.15.78:0/2086028370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe414012e50 con 0x5654f3f9d240 2024-04-25T01:45:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe40bfff640 1 -- 172.21.15.78:0/2086028370 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe41c04e050 con 0x5654f3f9d240 2024-04-25T01:45:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe435de5740 10 client.15468 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe435de5740 1 -- 172.21.15.78:0/2086028370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654f3fb1190 con 0x5654f3f9d240 2024-04-25T01:45:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe40bfff640 1 -- 172.21.15.78:0/2086028370 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe414012e50 con 0x5654f3f9d240 2024-04-25T01:45:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.954+0000 7fe40bfff640 1 -- 172.21.15.78:0/2086028370 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe41c0035f0 con 0x5654f3f9d240 2024-04-25T01:45:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.955+0000 7fe435de5740 10 client.15468 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.955+0000 7fe435de5740 10 client.15468 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-25T01:45:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.955+0000 7fe435de5740 20 client.15468 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.955+0000 7fe435de5740 1 --2- 172.21.15.78:0/2086028370 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x5654f3fb38e0 0x5654f3fb4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.955+0000 7fe435de5740 4 client.15468 mds_command: new command op to 24392 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:45:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.955+0000 7fe435de5740 1 -- 172.21.15.78:0/2086028370 --> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654f3f09440 con 0x5654f3fb38e0 2024-04-25T01:45:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.955+0000 7fe430d95640 1 --2- 172.21.15.78:0/2086028370 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x5654f3fb38e0 0x5654f3fb4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.955+0000 7fe40bfff640 10 client.15468 ms_handle_connect on v2:172.21.15.152:6832/923579368 2024-04-25T01:45:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.955+0000 7fe430d95640 1 --2- 172.21.15.78:0/2086028370 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x5654f3fb38e0 0x5654f3fb4d50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=226260de480240bd server_cookie=f1496e0ed28f46f9 in_seq=0 out_seq=0 2024-04-25T01:45:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.956+0000 7fe40bfff640 1 -- 172.21.15.78:0/2086028370 <== mds.3 v2:172.21.15.152:6832/923579368 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654f3f09440 con 0x5654f3fb38e0 2024-04-25T01:45:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.956+0000 7fe40bfff640 10 client.15468 handle_command_reply: tid=0 2024-04-25T01:45:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.956+0000 7fe435de5740 1 client.15468 shutdown 2024-04-25T01:45:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.956+0000 7fe435de5740 1 -- 172.21.15.78:0/2086028370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3f9d240 msgr2=0x5654f3fa7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.956+0000 7fe435de5740 1 --2- 172.21.15.78:0/2086028370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3f9d240 0x5654f3fa7100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe41c031610 tx=0x7fe41c0318e0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.956+0000 7fe435de5740 1 -- 172.21.15.78:0/2086028370 shutdown_connections 2024-04-25T01:45:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.956+0000 7fe435de5740 1 --2- 172.21.15.78:0/2086028370 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x5654f3fb38e0 0x5654f3fb4d50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.956+0000 7fe435de5740 1 --2- 172.21.15.78:0/2086028370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3fa23b0 0x5654f3fae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.956+0000 7fe40bfff640 0 client.15468 ms_handle_reset on v2:172.21.15.152:6832/923579368 2024-04-25T01:45:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.956+0000 7fe435de5740 1 --2- 172.21.15.78:0/2086028370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3f9d240 0x5654f3fa7100 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.956+0000 7fe435de5740 1 --2- 172.21.15.78:0/2086028370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9c850 0x5654f3fa6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.956+0000 7fe435de5740 1 -- 172.21.15.78:0/2086028370 >> 172.21.15.78:0/2086028370 conn(0x5654f3f97ec0 msgr2=0x5654f3f92ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.956+0000 7fe435de5740 1 -- 172.21.15.78:0/2086028370 shutdown_connections 2024-04-25T01:45:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.957+0000 7fe435de5740 1 -- 172.21.15.78:0/2086028370 wait complete. 2024-04-25T01:45:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.957+0000 7fe435de5740 20 client.15468 trim_cache size 0 max 16384 2024-04-25T01:45:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.974+0000 7fe435de5740 1 Processor -- start 2024-04-25T01:45:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.974+0000 7fe435de5740 1 -- start start 2024-04-25T01:45:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe435de5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3f9c850 0x5654f3f8e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe435de5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9d240 0x5654f3fae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe435de5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3fa23b0 0x5654f3fa6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe435de5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5654f3fb3090 con 0x5654f3fa23b0 2024-04-25T01:45:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe435de5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5654f3fb3760 con 0x5654f3f9c850 2024-04-25T01:45:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe435de5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654f3fb3e30 con 0x5654f3f9d240 2024-04-25T01:45:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe431596640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3fa23b0 0x5654f3fa6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe431596640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3fa23b0 0x5654f3fa6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60262/0 (socket says 172.21.15.78:60262) 2024-04-25T01:45:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe431596640 1 -- 172.21.15.78:0/2288738184 learned_addr learned my addr 172.21.15.78:0/2288738184 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe431596640 1 -- 172.21.15.78:0/2288738184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9d240 msgr2=0x5654f3fae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe431596640 1 --2- 172.21.15.78:0/2288738184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9d240 0x5654f3fae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe431596640 1 -- 172.21.15.78:0/2288738184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3f9c850 msgr2=0x5654f3f8e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe430d95640 1 --2- 172.21.15.78:0/2288738184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3f9c850 0x5654f3f8e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe431596640 1 --2- 172.21.15.78:0/2288738184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3f9c850 0x5654f3f8e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe431596640 1 -- 172.21.15.78:0/2288738184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654f3f9d620 con 0x5654f3fa23b0 2024-04-25T01:45:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe431596640 1 --2- 172.21.15.78:0/2288738184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3fa23b0 0x5654f3fa6bc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe414012620 tx=0x7fe414010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c558b4e8f03ae145 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe4227fc640 1 -- 172.21.15.78:0/2288738184 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41402d070 con 0x5654f3fa23b0 2024-04-25T01:45:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe4227fc640 1 -- 172.21.15.78:0/2288738184 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe414012e50 con 0x5654f3fa23b0 2024-04-25T01:45:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.975+0000 7fe4227fc640 1 -- 172.21.15.78:0/2288738184 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414024070 con 0x5654f3fa23b0 2024-04-25T01:45:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.976+0000 7fe435de5740 1 -- 172.21.15.78:0/2288738184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3fa23b0 msgr2=0x5654f3fa6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.976+0000 7fe435de5740 1 --2- 172.21.15.78:0/2288738184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3fa23b0 0x5654f3fa6bc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe414012620 tx=0x7fe414010f90 comp rx=0 tx=0).stop 2024-04-25T01:45:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.976+0000 7fe435de5740 1 -- 172.21.15.78:0/2288738184 shutdown_connections 2024-04-25T01:45:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.976+0000 7fe435de5740 1 --2- 172.21.15.78:0/2288738184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3fa23b0 0x5654f3fa6bc0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.976+0000 7fe435de5740 1 --2- 172.21.15.78:0/2288738184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9d240 0x5654f3fae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.976+0000 7fe435de5740 1 --2- 172.21.15.78:0/2288738184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3f9c850 0x5654f3f8e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.976+0000 7fe435de5740 1 -- 172.21.15.78:0/2288738184 >> 172.21.15.78:0/2288738184 conn(0x5654f3f97ec0 msgr2=0x5654f3f982a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.976+0000 7fe435de5740 1 -- 172.21.15.78:0/2288738184 shutdown_connections 2024-04-25T01:45:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.976+0000 7fe435de5740 1 -- 172.21.15.78:0/2288738184 wait complete. 2024-04-25T01:45:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.976+0000 7fe435de5740 1 -- start start 2024-04-25T01:45:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.976+0000 7fe435de5740 1 Processor -- start 2024-04-25T01:45:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe435de5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3f9c850 0x5654f3faa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe435de5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9d240 0x5654f3faa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe435de5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3fa23b0 0x5654f3fad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe435de5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5654f3fb4bd0 con 0x5654f3f9c850 2024-04-25T01:45:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe435de5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5654f3fb4500 con 0x5654f3fa23b0 2024-04-25T01:45:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe435de5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5654f3fb52a0 con 0x5654f3f9d240 2024-04-25T01:45:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe435de5740 10 client.0 _ll_register_callbacks cb 0x7fe41c032e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe430d95640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3f9c850 0x5654f3faa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe430d95640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3f9c850 0x5654f3faa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60268/0 (socket says 172.21.15.78:60268) 2024-04-25T01:45:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe430d95640 1 -- 172.21.15.78:0/2127477086 learned_addr learned my addr 172.21.15.78:0/2127477086 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe42bfff640 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9d240 0x5654f3faa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe430d95640 1 -- 172.21.15.78:0/2127477086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9d240 msgr2=0x5654f3faa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe430d95640 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9d240 0x5654f3faa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe431596640 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3fa23b0 0x5654f3fad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe430d95640 1 -- 172.21.15.78:0/2127477086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3fa23b0 msgr2=0x5654f3fad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:32.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe430d95640 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3fa23b0 0x5654f3fad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe430d95640 1 -- 172.21.15.78:0/2127477086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654f3fad890 con 0x5654f3f9c850 2024-04-25T01:45:32.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe42bfff640 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9d240 0x5654f3faa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:32.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe431596640 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3fa23b0 0x5654f3fad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:32.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe40bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:32.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe430d95640 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3f9c850 0x5654f3faa330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe424000f80 tx=0x7fe42401a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941af0c619014eb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:32.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe40bfff640 1 -- 172.21.15.78:0/2127477086 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4240a0250 con 0x5654f3f9c850 2024-04-25T01:45:32.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.977+0000 7fe40bfff640 1 -- 172.21.15.78:0/2127477086 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fe424099f40 con 0x5654f3f9c850 2024-04-25T01:45:32.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.978+0000 7fe435de5740 1 -- 172.21.15.78:0/2127477086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe414012e50 con 0x5654f3f9c850 2024-04-25T01:45:32.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.978+0000 7fe40bfff640 1 -- 172.21.15.78:0/2127477086 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4240a4b10 con 0x5654f3f9c850 2024-04-25T01:45:32.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.978+0000 7fe435de5740 10 client.15474 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:32.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.978+0000 7fe435de5740 1 -- 172.21.15.78:0/2127477086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654f4019eb0 con 0x5654f3f9c850 2024-04-25T01:45:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.978+0000 7fe40bfff640 1 -- 172.21.15.78:0/2127477086 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe414012e50 con 0x5654f3f9c850 2024-04-25T01:45:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.978+0000 7fe40bfff640 1 -- 172.21.15.78:0/2127477086 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe424022030 con 0x5654f3f9c850 2024-04-25T01:45:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.978+0000 7fe435de5740 10 client.15474 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.978+0000 7fe435de5740 10 client.15474 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-25T01:45:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.978+0000 7fe435de5740 20 client.15474 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.978+0000 7fe435de5740 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x5654f401eb10 0x5654f401d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.978+0000 7fe435de5740 4 client.15474 mds_command: new command op to 24392 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:45:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.978+0000 7fe435de5740 1 -- 172.21.15.78:0/2127477086 --> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] -- command(tid 0: {"prefix": "status"}) -- 0x5654f3f03790 con 0x5654f401eb10 2024-04-25T01:45:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.978+0000 7fe42bfff640 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x5654f401eb10 0x5654f401d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe42bfff640 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x5654f401eb10 0x5654f401d820 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=96e6a601a5a168e9 server_cookie=c973c8667b07c042 in_seq=0 out_seq=0 2024-04-25T01:45:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe40bfff640 10 client.15474 ms_handle_connect on v2:172.21.15.152:6832/923579368 2024-04-25T01:45:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe40bfff640 1 -- 172.21.15.78:0/2127477086 <== mds.3 v2:172.21.15.152:6832/923579368 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5654f3f03790 con 0x5654f401eb10 2024-04-25T01:45:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe40bfff640 10 client.15474 handle_command_reply: tid=0 2024-04-25T01:45:32.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe435de5740 1 client.15474 shutdown 2024-04-25T01:45:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe435de5740 1 -- 172.21.15.78:0/2127477086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3f9c850 msgr2=0x5654f3faa330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe435de5740 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3f9c850 0x5654f3faa330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe424000f80 tx=0x7fe42401a400 comp rx=0 tx=0).stop 2024-04-25T01:45:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe435de5740 1 -- 172.21.15.78:0/2127477086 shutdown_connections 2024-04-25T01:45:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe435de5740 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.152:6832/923579368,v1:172.21.15.152:6833/923579368] conn(0x5654f401eb10 0x5654f401d820 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe435de5740 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5654f3fa23b0 0x5654f3fad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe40bfff640 0 client.15474 ms_handle_reset on v2:172.21.15.152:6832/923579368 2024-04-25T01:45:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe435de5740 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5654f3f9d240 0x5654f3faa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe435de5740 1 --2- 172.21.15.78:0/2127477086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5654f3f9c850 0x5654f3faa330 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe435de5740 1 -- 172.21.15.78:0/2127477086 >> 172.21.15.78:0/2127477086 conn(0x5654f3f97ec0 msgr2=0x5654f3f8e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe435de5740 1 -- 172.21.15.78:0/2127477086 shutdown_connections 2024-04-25T01:45:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe435de5740 1 -- 172.21.15.78:0/2127477086 wait complete. 2024-04-25T01:45:32.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.979+0000 7fe435de5740 20 client.15474 trim_cache size 0 max 16384 2024-04-25T01:45:32.981 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:45:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "df36bea8-02a3-11ef-bc93-c7b262605968", 2024-04-25T01:45:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 3, 2024-04-25T01:45:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24392, 2024-04-25T01:45:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T01:45:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T01:45:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T01:45:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 110.562963786, 2024-04-25T01:45:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-04-25T01:45:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 94, 2024-04-25T01:45:32.981 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T01:45:32.982 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 169.978139137 2024-04-25T01:45:32.982 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:45:32.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.980+0000 7fe432ff9640 1 -- 172.21.15.78:0/2589330404 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4180a2b90 msgr2=0x7fe4180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.980+0000 7fe432ff9640 1 --2- 172.21.15.78:0/2589330404 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4180a2b90 0x7fe4180a5050 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe41400b590 tx=0x7fe41400baa0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.980+0000 7fe432ff9640 1 -- 172.21.15.78:0/2589330404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c101790 msgr2=0x7fe42c197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:32.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.980+0000 7fe432ff9640 1 --2- 172.21.15.78:0/2589330404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c101790 0x7fe42c197ac0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe42400ea80 tx=0x7fe42400ef40 comp rx=0 tx=0).stop 2024-04-25T01:45:32.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.980+0000 7fe432ff9640 1 -- 172.21.15.78:0/2589330404 shutdown_connections 2024-04-25T01:45:32.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.980+0000 7fe432ff9640 1 --2- 172.21.15.78:0/2589330404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe42c103340 0x7fe42c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.980+0000 7fe432ff9640 1 --2- 172.21.15.78:0/2589330404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe42c102980 0x7fe42c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.980+0000 7fe432ff9640 1 --2- 172.21.15.78:0/2589330404 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fe4180a2b90 0x7fe4180a5050 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.980+0000 7fe432ff9640 1 --2- 172.21.15.78:0/2589330404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe42c101790 0x7fe42c197ac0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:32.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.980+0000 7fe432ff9640 1 -- 172.21.15.78:0/2589330404 >> 172.21.15.78:0/2589330404 conn(0x7fe42c0fcd20 msgr2=0x7fe42c105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:32.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.980+0000 7fe432ff9640 1 -- 172.21.15.78:0/2589330404 shutdown_connections 2024-04-25T01:45:32.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:32.980+0000 7fe432ff9640 1 -- 172.21.15.78:0/2589330404 wait complete. 2024-04-25T01:45:32.985 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T01:45:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.093+0000 7f4c83790640 1 -- 172.21.15.78:0/1149195123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c7c075780 msgr2=0x7f4c7c073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.093+0000 7f4c83790640 1 --2- 172.21.15.78:0/1149195123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c7c075780 0x7f4c7c073bd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009900 tx=0x7f4c7802a410 comp rx=0 tx=0).stop 2024-04-25T01:45:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.093+0000 7f4c83790640 1 -- 172.21.15.78:0/1149195123 shutdown_connections 2024-04-25T01:45:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.093+0000 7f4c83790640 1 --2- 172.21.15.78:0/1149195123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c7c109640 0x7f4c7c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.093+0000 7f4c83790640 1 --2- 172.21.15.78:0/1149195123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c7c0741a0 0x7f4c7c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.093+0000 7f4c83790640 1 --2- 172.21.15.78:0/1149195123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c7c075780 0x7f4c7c073bd0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.093+0000 7f4c83790640 1 -- 172.21.15.78:0/1149195123 >> 172.21.15.78:0/1149195123 conn(0x7f4c7c0fccf0 msgr2=0x7f4c7c0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.093+0000 7f4c83790640 1 -- 172.21.15.78:0/1149195123 shutdown_connections 2024-04-25T01:45:33.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.093+0000 7f4c83790640 1 -- 172.21.15.78:0/1149195123 wait complete. 2024-04-25T01:45:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.093+0000 7f4c83790640 1 Processor -- start 2024-04-25T01:45:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.093+0000 7f4c83790640 1 -- start start 2024-04-25T01:45:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c83790640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c7c0741a0 0x7f4c7c199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c83790640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c7c075780 0x7f4c7c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c83790640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c7c109640 0x7f4c7c1a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c83790640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c10f5c0 con 0x7f4c7c0741a0 2024-04-25T01:45:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c83790640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c10f440 con 0x7f4c7c109640 2024-04-25T01:45:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c83790640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c10f740 con 0x7f4c7c075780 2024-04-25T01:45:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c8152c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c7c0741a0 0x7f4c7c199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c8152c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c7c0741a0 0x7f4c7c199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60288/0 (socket says 172.21.15.78:60288) 2024-04-25T01:45:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c8152c640 1 -- 172.21.15.78:0/3582357480 learned_addr learned my addr 172.21.15.78:0/3582357480 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c80d2b640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c7c075780 0x7f4c7c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c81d2d640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c7c109640 0x7f4c7c1a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c8152c640 1 -- 172.21.15.78:0/3582357480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c7c075780 msgr2=0x7f4c7c19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c8152c640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c7c075780 0x7f4c7c19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c8152c640 1 -- 172.21.15.78:0/3582357480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c7c109640 msgr2=0x7f4c7c1a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c8152c640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c7c109640 0x7f4c7c1a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c8152c640 1 -- 172.21.15.78:0/3582357480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7c1a1910 con 0x7f4c7c0741a0 2024-04-25T01:45:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c81d2d640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c7c109640 0x7f4c7c1a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c8152c640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c7c0741a0 0x7f4c7c199d00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009230 tx=0x7f4c78004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c727fc640 1 -- 172.21.15.78:0/3582357480 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7803f030 con 0x7f4c7c0741a0 2024-04-25T01:45:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c83790640 1 -- 172.21.15.78:0/3582357480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7c1a1ba0 con 0x7f4c7c0741a0 2024-04-25T01:45:33.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c727fc640 1 -- 172.21.15.78:0/3582357480 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c78003ce0 con 0x7f4c7c0741a0 2024-04-25T01:45:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.094+0000 7f4c727fc640 1 -- 172.21.15.78:0/3582357480 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78035070 con 0x7f4c7c0741a0 2024-04-25T01:45:33.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.095+0000 7f4c83790640 1 -- 172.21.15.78:0/3582357480 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c7c1a20e0 con 0x7f4c7c0741a0 2024-04-25T01:45:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c727fc640 1 -- 172.21.15.78:0/3582357480 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4c78042030 con 0x7f4c7c0741a0 2024-04-25T01:45:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c86578740 1 Processor -- start 2024-04-25T01:45:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c86578740 1 -- start start 2024-04-25T01:45:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c86578740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2440 0x55d8566e2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c86578740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e2e30 0x55d8566e7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c86578740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e7fa0 0x55d8566ea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c86578740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d856645f00 con 0x55d8566e2440 2024-04-25T01:45:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c86578740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d8565ee750 con 0x55d8566e2e30 2024-04-25T01:45:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c86578740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d856641ec0 con 0x55d8566e7fa0 2024-04-25T01:45:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c81d2d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2440 0x55d8566e2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c81d2d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2440 0x55d8566e2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60294/0 (socket says 172.21.15.78:60294) 2024-04-25T01:45:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c81d2d640 1 -- 172.21.15.78:0/1673935183 learned_addr learned my addr 172.21.15.78:0/1673935183 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c8152c640 1 --2- 172.21.15.78:0/1673935183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e2e30 0x55d8566e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c80d2b640 1 --2- 172.21.15.78:0/1673935183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e7fa0 0x55d8566ea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.097+0000 7f4c727fc640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4c680a2b90 0x7f4c680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.098+0000 7f4c81d2d640 1 -- 172.21.15.78:0/1673935183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e7fa0 msgr2=0x55d8566ea430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.098+0000 7f4c81d2d640 1 --2- 172.21.15.78:0/1673935183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e7fa0 0x55d8566ea430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.098+0000 7f4c81d2d640 1 -- 172.21.15.78:0/1673935183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e2e30 msgr2=0x55d8566e7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.098+0000 7f4c81d2d640 1 --2- 172.21.15.78:0/1673935183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e2e30 0x55d8566e7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.098+0000 7f4c81d2d640 1 -- 172.21.15.78:0/1673935183 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d85635a2d0 con 0x55d8566e2440 2024-04-25T01:45:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.098+0000 7f4c81d2d640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4c680a2b90 0x7f4c680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.098+0000 7f4c727fc640 1 -- 172.21.15.78:0/3582357480 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f4c780efed0 con 0x7f4c7c0741a0 2024-04-25T01:45:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.098+0000 7f4c81d2d640 1 --2- 172.21.15.78:0/1673935183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2440 0x55d8566e2860 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400f530 tx=0x7f4c7400f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df27b41fd2b29eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.098+0000 7f4c677fe640 1 -- 172.21.15.78:0/1673935183 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7401c400 con 0x55d8566e2440 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.098+0000 7f4c677fe640 1 -- 172.21.15.78:0/1673935183 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c7401ca80 con 0x55d8566e2440 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.098+0000 7f4c677fe640 1 -- 172.21.15.78:0/1673935183 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400d980 con 0x55d8566e2440 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.099+0000 7f4c86578740 1 -- 172.21.15.78:0/1673935183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2440 msgr2=0x55d8566e2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.099+0000 7f4c81d2d640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4c680a2b90 0x7f4c680a5050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4c74020000 tx=0x7f4c7401d000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.099+0000 7f4c86578740 1 --2- 172.21.15.78:0/1673935183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2440 0x55d8566e2860 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400f530 tx=0x7f4c7400f9f0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.099+0000 7f4c86578740 1 -- 172.21.15.78:0/1673935183 shutdown_connections 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.099+0000 7f4c86578740 1 --2- 172.21.15.78:0/1673935183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e7fa0 0x55d8566ea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.099+0000 7f4c86578740 1 --2- 172.21.15.78:0/1673935183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e2e30 0x55d8566e7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.099+0000 7f4c86578740 1 --2- 172.21.15.78:0/1673935183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2440 0x55d8566e2860 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.099+0000 7f4c86578740 1 -- 172.21.15.78:0/1673935183 >> 172.21.15.78:0/1673935183 conn(0x55d8566ddab0 msgr2=0x55d8566ddeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.099+0000 7f4c86578740 1 -- 172.21.15.78:0/1673935183 shutdown_connections 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.099+0000 7f4c86578740 1 -- 172.21.15.78:0/1673935183 wait complete. 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.099+0000 7f4c86578740 1 -- start start 2024-04-25T01:45:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.099+0000 7f4c86578740 1 Processor -- start 2024-04-25T01:45:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c86578740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2440 0x55d8566ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c86578740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e2e30 0x55d8566ecd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c86578740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e7fa0 0x55d8566f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c86578740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d8566eaf50 con 0x55d8566e2440 2024-04-25T01:45:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c86578740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d856601cf0 con 0x55d8566e2e30 2024-04-25T01:45:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c86578740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d8566eb0d0 con 0x55d8566e7fa0 2024-04-25T01:45:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c86578740 10 client.0 _ll_register_callbacks cb 0x55d8566016e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c8152c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2440 0x55d8566ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c8152c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2440 0x55d8566ec7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60306/0 (socket says 172.21.15.78:60306) 2024-04-25T01:45:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c8152c640 1 -- 172.21.15.78:0/3422092455 learned_addr learned my addr 172.21.15.78:0/3422092455 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c8152c640 1 -- 172.21.15.78:0/3422092455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e7fa0 msgr2=0x55d8566f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c8152c640 1 --2- 172.21.15.78:0/3422092455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e7fa0 0x55d8566f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c8152c640 1 -- 172.21.15.78:0/3422092455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e2e30 msgr2=0x55d8566ecd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c8152c640 1 --2- 172.21.15.78:0/3422092455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e2e30 0x55d8566ecd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c8152c640 1 -- 172.21.15.78:0/3422092455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8564246b0 con 0x55d8566e2440 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c8152c640 1 --2- 172.21.15.78:0/3422092455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2440 0x55d8566ec7d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009a20 tx=0x7f4c780ba360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb52713355ca436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c64ff9640 1 -- 172.21.15.78:0/3422092455 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78049040 con 0x55d8566e2440 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c64ff9640 1 -- 172.21.15.78:0/3422092455 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f4c780c0480 con 0x55d8566e2440 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c86578740 1 -- 172.21.15.78:0/3422092455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c7401ca80 con 0x55d8566e2440 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c64ff9640 1 -- 172.21.15.78:0/3422092455 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c780bbe90 con 0x55d8566e2440 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c86578740 10 client.15486 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c86578740 1 -- 172.21.15.78:0/3422092455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8566f6d30 con 0x55d8566e2440 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.100+0000 7f4c64ff9640 1 -- 172.21.15.78:0/3422092455 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c7401ca80 con 0x55d8566e2440 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.101+0000 7f4c64ff9640 1 -- 172.21.15.78:0/3422092455 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c780c4b30 con 0x55d8566e2440 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.101+0000 7f4c86578740 10 client.15486 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.101+0000 7f4c86578740 10 client.15486 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T01:45:33.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.101+0000 7f4c86578740 20 client.15486 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.101+0000 7f4c86578740 1 --2- 172.21.15.78:0/3422092455 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55d8566f9520 0x55d8566fa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.101+0000 7f4c86578740 4 client.15486 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:45:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.101+0000 7f4c86578740 1 -- 172.21.15.78:0/3422092455 --> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d856653440 con 0x55d8566f9520 2024-04-25T01:45:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.101+0000 7f4c80d2b640 1 --2- 172.21.15.78:0/3422092455 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55d8566f9520 0x55d8566fa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.101+0000 7f4c80d2b640 1 --2- 172.21.15.78:0/3422092455 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55d8566f9520 0x55d8566fa990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94616b2a3e021d88 server_cookie=c406f909bb7732cf in_seq=0 out_seq=0 2024-04-25T01:45:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.101+0000 7f4c64ff9640 10 client.15486 ms_handle_connect on v2:172.21.15.196:6838/460829167 2024-04-25T01:45:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.102+0000 7f4c64ff9640 1 -- 172.21.15.78:0/3422092455 <== mds.2 v2:172.21.15.196:6838/460829167 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d856653440 con 0x55d8566f9520 2024-04-25T01:45:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.102+0000 7f4c64ff9640 10 client.15486 handle_command_reply: tid=0 2024-04-25T01:45:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.102+0000 7f4c86578740 1 client.15486 shutdown 2024-04-25T01:45:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.102+0000 7f4c86578740 1 -- 172.21.15.78:0/3422092455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2440 msgr2=0x55d8566ec7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.102+0000 7f4c86578740 1 --2- 172.21.15.78:0/3422092455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2440 0x55d8566ec7d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009a20 tx=0x7f4c780ba360 comp rx=0 tx=0).stop 2024-04-25T01:45:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.102+0000 7f4c86578740 1 -- 172.21.15.78:0/3422092455 shutdown_connections 2024-04-25T01:45:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.102+0000 7f4c86578740 1 --2- 172.21.15.78:0/3422092455 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55d8566f9520 0x55d8566fa990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.102+0000 7f4c86578740 1 --2- 172.21.15.78:0/3422092455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e7fa0 0x55d8566f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.102+0000 7f4c86578740 1 --2- 172.21.15.78:0/3422092455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e2e30 0x55d8566ecd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.102+0000 7f4c64ff9640 0 client.15486 ms_handle_reset on v2:172.21.15.196:6838/460829167 2024-04-25T01:45:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.102+0000 7f4c86578740 1 --2- 172.21.15.78:0/3422092455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2440 0x55d8566ec7d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.102+0000 7f4c86578740 1 -- 172.21.15.78:0/3422092455 >> 172.21.15.78:0/3422092455 conn(0x55d8566ddab0 msgr2=0x55d8566dcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.102+0000 7f4c86578740 1 -- 172.21.15.78:0/3422092455 shutdown_connections 2024-04-25T01:45:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.102+0000 7f4c86578740 1 -- 172.21.15.78:0/3422092455 wait complete. 2024-04-25T01:45:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.103+0000 7f4c86578740 20 client.15486 trim_cache size 0 max 16384 2024-04-25T01:45:33.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.120+0000 7f4c86578740 1 Processor -- start 2024-04-25T01:45:33.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.120+0000 7f4c86578740 1 -- start start 2024-04-25T01:45:33.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.120+0000 7f4c86578740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e2440 0x55d8566f8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.120+0000 7f4c86578740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2e30 0x55d8566f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.120+0000 7f4c86578740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e7fa0 0x55d8566f85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.120+0000 7f4c86578740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d8566f8cd0 con 0x55d8566e2e30 2024-04-25T01:45:33.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.120+0000 7f4c86578740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d8566f93a0 con 0x55d8566e7fa0 2024-04-25T01:45:33.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.120+0000 7f4c86578740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d8566f9a70 con 0x55d8566e2440 2024-04-25T01:45:33.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.120+0000 7f4c80d2b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2e30 0x55d8566f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.120+0000 7f4c80d2b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2e30 0x55d8566f3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60314/0 (socket says 172.21.15.78:60314) 2024-04-25T01:45:33.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.120+0000 7f4c80d2b640 1 -- 172.21.15.78:0/1567252262 learned_addr learned my addr 172.21.15.78:0/1567252262 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.120+0000 7f4c8152c640 1 --2- 172.21.15.78:0/1567252262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e2440 0x55d8566f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.120+0000 7f4c80d2b640 1 -- 172.21.15.78:0/1567252262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e2440 msgr2=0x55d8566f8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c80d2b640 1 --2- 172.21.15.78:0/1567252262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e2440 0x55d8566f8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c80d2b640 1 -- 172.21.15.78:0/1567252262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e7fa0 msgr2=0x55d8566f85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c80d2b640 1 --2- 172.21.15.78:0/1567252262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e7fa0 0x55d8566f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c80d2b640 1 -- 172.21.15.78:0/1567252262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8566f8b00 con 0x55d8566e2e30 2024-04-25T01:45:33.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c8152c640 1 --2- 172.21.15.78:0/1567252262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e2440 0x55d8566f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:33.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c80d2b640 1 --2- 172.21.15.78:0/1567252262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2e30 0x55d8566f3d10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c000c00 tx=0x7f4c6c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf22990a9454433 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c66ffd640 1 -- 172.21.15.78:0/1567252262 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c01e070 con 0x55d8566e2e30 2024-04-25T01:45:33.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c66ffd640 1 -- 172.21.15.78:0/1567252262 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c003f80 con 0x55d8566e2e30 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c66ffd640 1 -- 172.21.15.78:0/1567252262 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c01a520 con 0x55d8566e2e30 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c86578740 1 -- 172.21.15.78:0/1567252262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2e30 msgr2=0x55d8566f3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c86578740 1 --2- 172.21.15.78:0/1567252262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2e30 0x55d8566f3d10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c000c00 tx=0x7f4c6c00bed0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c86578740 1 -- 172.21.15.78:0/1567252262 shutdown_connections 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c86578740 1 --2- 172.21.15.78:0/1567252262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e7fa0 0x55d8566f85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c86578740 1 --2- 172.21.15.78:0/1567252262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e2e30 0x55d8566f3d10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c86578740 1 --2- 172.21.15.78:0/1567252262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e2440 0x55d8566f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c86578740 1 -- 172.21.15.78:0/1567252262 >> 172.21.15.78:0/1567252262 conn(0x55d8566ddab0 msgr2=0x55d8566e36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c86578740 1 -- 172.21.15.78:0/1567252262 shutdown_connections 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.121+0000 7f4c86578740 1 -- 172.21.15.78:0/1567252262 wait complete. 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.122+0000 7f4c86578740 1 -- start start 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.122+0000 7f4c86578740 1 Processor -- start 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.122+0000 7f4c86578740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e2440 0x55d8566eff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.122+0000 7f4c86578740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e2e30 0x55d8566f04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.122+0000 7f4c86578740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e7fa0 0x55d8566f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.122+0000 7f4c86578740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d8566fa810 con 0x55d8566e7fa0 2024-04-25T01:45:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.122+0000 7f4c86578740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d8566fa140 con 0x55d8566e2440 2024-04-25T01:45:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.122+0000 7f4c86578740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d8566faee0 con 0x55d8566e2e30 2024-04-25T01:45:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.122+0000 7f4c86578740 10 client.0 _ll_register_callbacks cb 0x7f4c780f36a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.122+0000 7f4c81d2d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e7fa0 0x55d8566f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.122+0000 7f4c81d2d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e7fa0 0x55d8566f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60318/0 (socket says 172.21.15.78:60318) 2024-04-25T01:45:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.122+0000 7f4c81d2d640 1 -- 172.21.15.78:0/200303596 learned_addr learned my addr 172.21.15.78:0/200303596 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.122+0000 7f4c8152c640 1 --2- 172.21.15.78:0/200303596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e2440 0x55d8566eff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.122+0000 7f4c80d2b640 1 --2- 172.21.15.78:0/200303596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e2e30 0x55d8566f04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c81d2d640 1 -- 172.21.15.78:0/200303596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e2e30 msgr2=0x55d8566f04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c81d2d640 1 --2- 172.21.15.78:0/200303596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e2e30 0x55d8566f04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c81d2d640 1 -- 172.21.15.78:0/200303596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e2440 msgr2=0x55d8566eff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c81d2d640 1 --2- 172.21.15.78:0/200303596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e2440 0x55d8566eff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c81d2d640 1 -- 172.21.15.78:0/200303596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8566f35c0 con 0x55d8566e7fa0 2024-04-25T01:45:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c81d2d640 1 --2- 172.21.15.78:0/200303596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e7fa0 0x55d8566f2f80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400f500 tx=0x7f4c740126e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6789b68ae63997a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c64ff9640 1 -- 172.21.15.78:0/200303596 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400dca0 con 0x55d8566e7fa0 2024-04-25T01:45:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c64ff9640 1 -- 172.21.15.78:0/200303596 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f4c740049e0 con 0x55d8566e7fa0 2024-04-25T01:45:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c86578740 1 -- 172.21.15.78:0/200303596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c6c003f80 con 0x55d8566e7fa0 2024-04-25T01:45:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c64ff9640 1 -- 172.21.15.78:0/200303596 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c7403c050 con 0x55d8566e7fa0 2024-04-25T01:45:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c86578740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c86578740 1 -- 172.21.15.78:0/200303596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d85675fa90 con 0x55d8566e7fa0 2024-04-25T01:45:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c64ff9640 1 -- 172.21.15.78:0/200303596 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c003f80 con 0x55d8566e7fa0 2024-04-25T01:45:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c64ff9640 1 -- 172.21.15.78:0/200303596 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c74037030 con 0x55d8566e7fa0 2024-04-25T01:45:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c86578740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c86578740 10 client.15492 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T01:45:33.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c86578740 20 client.15492 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c86578740 1 --2- 172.21.15.78:0/200303596 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55d856764470 0x55d856763180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c86578740 4 client.15492 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.123+0000 7f4c86578740 1 -- 172.21.15.78:0/200303596 --> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] -- command(tid 0: {"prefix": "status"}) -- 0x55d85664d790 con 0x55d856764470 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c8152c640 1 --2- 172.21.15.78:0/200303596 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55d856764470 0x55d856763180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c64ff9640 10 client.15492 ms_handle_connect on v2:172.21.15.196:6838/460829167 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c8152c640 1 --2- 172.21.15.78:0/200303596 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55d856764470 0x55d856763180 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c956abff8d14d55e server_cookie=66bae0c153096c68 in_seq=0 out_seq=0 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c64ff9640 1 -- 172.21.15.78:0/200303596 <== mds.2 v2:172.21.15.196:6838/460829167 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d85664d790 con 0x55d856764470 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c64ff9640 10 client.15492 handle_command_reply: tid=0 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c86578740 1 client.15492 shutdown 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c86578740 1 -- 172.21.15.78:0/200303596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e7fa0 msgr2=0x55d8566f2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c86578740 1 --2- 172.21.15.78:0/200303596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e7fa0 0x55d8566f2f80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400f500 tx=0x7f4c740126e0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c86578740 1 -- 172.21.15.78:0/200303596 shutdown_connections 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c86578740 1 --2- 172.21.15.78:0/200303596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8566e7fa0 0x55d8566f2f80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c86578740 1 --2- 172.21.15.78:0/200303596 >> [v2:172.21.15.196:6838/460829167,v1:172.21.15.196:6839/460829167] conn(0x55d856764470 0x55d856763180 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c86578740 1 --2- 172.21.15.78:0/200303596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8566e2e30 0x55d8566f04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c86578740 1 --2- 172.21.15.78:0/200303596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8566e2440 0x55d8566eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c64ff9640 0 client.15492 ms_handle_reset on v2:172.21.15.196:6838/460829167 2024-04-25T01:45:33.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.124+0000 7f4c86578740 1 -- 172.21.15.78:0/200303596 >> 172.21.15.78:0/200303596 conn(0x55d8566ddab0 msgr2=0x55d8566e41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c86578740 1 -- 172.21.15.78:0/200303596 shutdown_connections 2024-04-25T01:45:33.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c86578740 1 -- 172.21.15.78:0/200303596 wait complete. 2024-04-25T01:45:33.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c86578740 20 client.15492 trim_cache size 0 max 16384 2024-04-25T01:45:33.124 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:45:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "df36bea8-02a3-11ef-bc93-c7b262605968", 2024-04-25T01:45:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 2, 2024-04-25T01:45:33.124 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24448, 2024-04-25T01:45:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T01:45:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T01:45:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T01:45:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 111.12662908900001, 2024-04-25T01:45:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-04-25T01:45:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 94, 2024-04-25T01:45:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T01:45:33.125 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 149.74700352400001 2024-04-25T01:45:33.125 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:45:33.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c83790640 1 -- 172.21.15.78:0/3582357480 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4c680a2b90 msgr2=0x7f4c680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c83790640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4c680a2b90 0x7f4c680a5050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4c74020000 tx=0x7f4c7401d000 comp rx=0 tx=0).stop 2024-04-25T01:45:33.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c83790640 1 -- 172.21.15.78:0/3582357480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c7c0741a0 msgr2=0x7f4c7c199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c83790640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c7c0741a0 0x7f4c7c199d00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009230 tx=0x7f4c78004360 comp rx=0 tx=0).stop 2024-04-25T01:45:33.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c83790640 1 -- 172.21.15.78:0/3582357480 shutdown_connections 2024-04-25T01:45:33.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c83790640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c7c109640 0x7f4c7c1a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c83790640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c7c075780 0x7f4c7c19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c83790640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4c680a2b90 0x7f4c680a5050 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c83790640 1 --2- 172.21.15.78:0/3582357480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4c7c0741a0 0x7f4c7c199d00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c83790640 1 -- 172.21.15.78:0/3582357480 >> 172.21.15.78:0/3582357480 conn(0x7f4c7c0fccf0 msgr2=0x7f4c7c105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c83790640 1 -- 172.21.15.78:0/3582357480 shutdown_connections 2024-04-25T01:45:33.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.125+0000 7f4c83790640 1 -- 172.21.15.78:0/3582357480 wait complete. 2024-04-25T01:45:33.128 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T01:45:33.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.234+0000 7fcf11190640 1 -- 172.21.15.78:0/3500960549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0c102230 msgr2=0x7fcf0c108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.234+0000 7fcf11190640 1 --2- 172.21.15.78:0/3500960549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0c102230 0x7fcf0c108ac0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009ca0 tx=0x7fcf0402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.234+0000 7fcf11190640 1 -- 172.21.15.78:0/3500960549 shutdown_connections 2024-04-25T01:45:33.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.234+0000 7fcf11190640 1 --2- 172.21.15.78:0/3500960549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0c102230 0x7fcf0c108ac0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.234+0000 7fcf11190640 1 --2- 172.21.15.78:0/3500960549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf0c101870 0x7fcf0c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.234+0000 7fcf11190640 1 --2- 172.21.15.78:0/3500960549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf0c100680 0x7fcf0c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.234+0000 7fcf11190640 1 -- 172.21.15.78:0/3500960549 >> 172.21.15.78:0/3500960549 conn(0x7fcf0c0fbbf0 msgr2=0x7fcf0c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.234+0000 7fcf11190640 1 -- 172.21.15.78:0/3500960549 shutdown_connections 2024-04-25T01:45:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.234+0000 7fcf11190640 1 -- 172.21.15.78:0/3500960549 wait complete. 2024-04-25T01:45:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.235+0000 7fcf11190640 1 Processor -- start 2024-04-25T01:45:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.235+0000 7fcf11190640 1 -- start start 2024-04-25T01:45:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.235+0000 7fcf11190640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf0c100680 0x7fcf0c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.235+0000 7fcf11190640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf0c101870 0x7fcf0c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.235+0000 7fcf11190640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0c102230 0x7fcf0c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.235+0000 7fcf11190640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c10c2e0 con 0x7fcf0c102230 2024-04-25T01:45:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.235+0000 7fcf11190640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c10c160 con 0x7fcf0c100680 2024-04-25T01:45:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.235+0000 7fcf11190640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c10c460 con 0x7fcf0c101870 2024-04-25T01:45:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.235+0000 7fcf0b59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0c102230 0x7fcf0c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.235+0000 7fcf0b59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0c102230 0x7fcf0c19edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60332/0 (socket says 172.21.15.78:60332) 2024-04-25T01:45:33.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.235+0000 7fcf0b59e640 1 -- 172.21.15.78:0/708687126 learned_addr learned my addr 172.21.15.78:0/708687126 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.235+0000 7fcf0ad9d640 1 --2- 172.21.15.78:0/708687126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf0c100680 0x7fcf0c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.235+0000 7fcf0259c640 1 --2- 172.21.15.78:0/708687126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf0c101870 0x7fcf0c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.236+0000 7fcf0b59e640 1 -- 172.21.15.78:0/708687126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf0c101870 msgr2=0x7fcf0c19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.236+0000 7fcf0b59e640 1 --2- 172.21.15.78:0/708687126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf0c101870 0x7fcf0c19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.236+0000 7fcf0b59e640 1 -- 172.21.15.78:0/708687126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf0c100680 msgr2=0x7fcf0c069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.236+0000 7fcf0b59e640 1 --2- 172.21.15.78:0/708687126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf0c100680 0x7fcf0c069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.236+0000 7fcf0b59e640 1 -- 172.21.15.78:0/708687126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c19f680 con 0x7fcf0c102230 2024-04-25T01:45:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.236+0000 7fcf0b59e640 1 --2- 172.21.15.78:0/708687126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0c102230 0x7fcf0c19edc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcf04031ae0 tx=0x7fcf04031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.236+0000 7fcf08d99640 1 -- 172.21.15.78:0/708687126 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04003c30 con 0x7fcf0c102230 2024-04-25T01:45:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.236+0000 7fcf11190640 1 -- 172.21.15.78:0/708687126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0c19f910 con 0x7fcf0c102230 2024-04-25T01:45:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.236+0000 7fcf08d99640 1 -- 172.21.15.78:0/708687126 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf04003dd0 con 0x7fcf0c102230 2024-04-25T01:45:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.236+0000 7fcf08d99640 1 -- 172.21.15.78:0/708687126 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0402fcb0 con 0x7fcf0c102230 2024-04-25T01:45:33.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.236+0000 7fcf11190640 1 -- 172.21.15.78:0/708687126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0c19feb0 con 0x7fcf0c102230 2024-04-25T01:45:33.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.238+0000 7fcf08d99640 1 -- 172.21.15.78:0/708687126 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcf04004050 con 0x7fcf0c102230 2024-04-25T01:45:33.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.238+0000 7fcf13f7f740 1 Processor -- start 2024-04-25T01:45:33.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.238+0000 7fcf13f7f740 1 -- start start 2024-04-25T01:45:33.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf13f7f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af737850 0x5565af737c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf13f7f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af738240 0x5565af73cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf13f7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af73d3b0 0x5565af73f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf13f7f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5565af696f00 con 0x5565af738240 2024-04-25T01:45:33.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf13f7f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5565af63f750 con 0x5565af737850 2024-04-25T01:45:33.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf13f7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565af692ec0 con 0x5565af73d3b0 2024-04-25T01:45:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf0ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af738240 0x5565af73cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf08d99640 1 --2- 172.21.15.78:0/708687126 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fcef80a2b90 0x7fcef80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf0ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af738240 0x5565af73cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60342/0 (socket says 172.21.15.78:60342) 2024-04-25T01:45:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf0ad9d640 1 -- 172.21.15.78:0/290895959 learned_addr learned my addr 172.21.15.78:0/290895959 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf0b59e640 1 --2- 172.21.15.78:0/708687126 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fcef80a2b90 0x7fcef80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf0259c640 1 --2- 172.21.15.78:0/290895959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af73d3b0 0x5565af73f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf0259c640 1 --2- 172.21.15.78:0/290895959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af737850 0x5565af737c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf08d99640 1 -- 172.21.15.78:0/708687126 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fcf040f0670 con 0x7fcf0c102230 2024-04-25T01:45:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf0ad9d640 1 -- 172.21.15.78:0/290895959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af73d3b0 msgr2=0x5565af73f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf0ad9d640 1 --2- 172.21.15.78:0/290895959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af73d3b0 0x5565af73f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf0ad9d640 1 -- 172.21.15.78:0/290895959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af737850 msgr2=0x5565af737c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf0ad9d640 1 --2- 172.21.15.78:0/290895959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af737850 0x5565af737c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf0ad9d640 1 -- 172.21.15.78:0/290895959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565af3ab2d0 con 0x5565af738240 2024-04-25T01:45:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.239+0000 7fcf0259c640 1 --2- 172.21.15.78:0/290895959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af737850 0x5565af737c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.240+0000 7fcf0ad9d640 1 --2- 172.21.15.78:0/290895959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af738240 0x5565af73cd40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00b7d0 tx=0x7fcefc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e703e4c9c537813 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.240+0000 7fcf0b59e640 1 --2- 172.21.15.78:0/708687126 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fcef80a2b90 0x7fcef80a5050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcf04082d10 tx=0x7fcf0403d040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.240+0000 7fcf0159a640 1 -- 172.21.15.78:0/290895959 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc003dd0 con 0x5565af738240 2024-04-25T01:45:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.240+0000 7fcf0159a640 1 -- 172.21.15.78:0/290895959 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcefc015070 con 0x5565af738240 2024-04-25T01:45:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.240+0000 7fcf0159a640 1 -- 172.21.15.78:0/290895959 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc011760 con 0x5565af738240 2024-04-25T01:45:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.240+0000 7fcf13f7f740 1 -- 172.21.15.78:0/290895959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af738240 msgr2=0x5565af73cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.240+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/290895959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af738240 0x5565af73cd40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00b7d0 tx=0x7fcefc004610 comp rx=0 tx=0).stop 2024-04-25T01:45:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 -- 172.21.15.78:0/290895959 shutdown_connections 2024-04-25T01:45:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/290895959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af73d3b0 0x5565af73f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/290895959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af738240 0x5565af73cd40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/290895959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af737850 0x5565af737c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 -- 172.21.15.78:0/290895959 >> 172.21.15.78:0/290895959 conn(0x5565af732ec0 msgr2=0x5565af7332c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 -- 172.21.15.78:0/290895959 shutdown_connections 2024-04-25T01:45:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 -- 172.21.15.78:0/290895959 wait complete. 2024-04-25T01:45:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 -- start start 2024-04-25T01:45:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 Processor -- start 2024-04-25T01:45:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af737850 0x5565af741c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af738240 0x5565af7421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af73d3b0 0x5565af749200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5565af740360 con 0x5565af737850 2024-04-25T01:45:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5565af652cf0 con 0x5565af738240 2024-04-25T01:45:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565af7404e0 con 0x5565af73d3b0 2024-04-25T01:45:33.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf13f7f740 10 client.0 _ll_register_callbacks cb 0x5565af6526e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf0259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af737850 0x5565af741c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf0259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af737850 0x5565af741c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60354/0 (socket says 172.21.15.78:60354) 2024-04-25T01:45:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf0259c640 1 -- 172.21.15.78:0/53644740 learned_addr learned my addr 172.21.15.78:0/53644740 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf0259c640 1 --2- 172.21.15.78:0/53644740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af73d3b0 0x5565af749200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.241+0000 7fcf0ad9d640 1 --2- 172.21.15.78:0/53644740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af738240 0x5565af7421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf0259c640 1 -- 172.21.15.78:0/53644740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af73d3b0 msgr2=0x5565af749200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf0259c640 1 --2- 172.21.15.78:0/53644740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af73d3b0 0x5565af749200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf0259c640 1 -- 172.21.15.78:0/53644740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af738240 msgr2=0x5565af7421c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf0259c640 1 --2- 172.21.15.78:0/53644740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af738240 0x5565af7421c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf0259c640 1 -- 172.21.15.78:0/53644740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565af4756b0 con 0x5565af737850 2024-04-25T01:45:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf0259c640 1 --2- 172.21.15.78:0/53644740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af737850 0x5565af741c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcef4010ac0 tx=0x7fcef4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b80c54238437bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcef27fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:33.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcef27fc640 1 -- 172.21.15.78:0/53644740 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef400fc90 con 0x5565af737850 2024-04-25T01:45:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcef27fc640 1 -- 172.21.15.78:0/53644740 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fcef400fe30 con 0x5565af737850 2024-04-25T01:45:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf13f7f740 1 -- 172.21.15.78:0/53644740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcefc015070 con 0x5565af737850 2024-04-25T01:45:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcef27fc640 1 -- 172.21.15.78:0/53644740 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcef4005020 con 0x5565af737850 2024-04-25T01:45:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf13f7f740 10 client.15504 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf13f7f740 1 -- 172.21.15.78:0/53644740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565af74c2b0 con 0x5565af737850 2024-04-25T01:45:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcef27fc640 1 -- 172.21.15.78:0/53644740 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcefc015070 con 0x5565af737850 2024-04-25T01:45:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcef27fc640 1 -- 172.21.15.78:0/53644740 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcef401d030 con 0x5565af737850 2024-04-25T01:45:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf13f7f740 10 client.15504 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf13f7f740 10 client.15504 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T01:45:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf13f7f740 20 client.15504 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/53644740 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x5565af74e8f0 0x5565af74fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf13f7f740 4 client.15504 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:45:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.242+0000 7fcf13f7f740 1 -- 172.21.15.78:0/53644740 --> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565af6a4440 con 0x5565af74e8f0 2024-04-25T01:45:33.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.243+0000 7fcf0ad9d640 1 --2- 172.21.15.78:0/53644740 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x5565af74e8f0 0x5565af74fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.243+0000 7fcef27fc640 10 client.15504 ms_handle_connect on v2:172.21.15.152:6836/4112159743 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.243+0000 7fcf0ad9d640 1 --2- 172.21.15.78:0/53644740 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x5565af74e8f0 0x5565af74fd60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=daf471f060fa721a server_cookie=eaf22aa155d4ba51 in_seq=0 out_seq=0 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcef27fc640 1 -- 172.21.15.78:0/53644740 <== mds.1 v2:172.21.15.152:6836/4112159743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565af6a4440 con 0x5565af74e8f0 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcef27fc640 10 client.15504 handle_command_reply: tid=0 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcf13f7f740 1 client.15504 shutdown 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcf13f7f740 1 -- 172.21.15.78:0/53644740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af737850 msgr2=0x5565af741c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/53644740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af737850 0x5565af741c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcef4010ac0 tx=0x7fcef4004730 comp rx=0 tx=0).stop 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcf13f7f740 1 -- 172.21.15.78:0/53644740 shutdown_connections 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/53644740 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x5565af74e8f0 0x5565af74fd60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/53644740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af73d3b0 0x5565af749200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/53644740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af738240 0x5565af7421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcef27fc640 0 client.15504 ms_handle_reset on v2:172.21.15.152:6836/4112159743 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/53644740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af737850 0x5565af741c80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcf13f7f740 1 -- 172.21.15.78:0/53644740 >> 172.21.15.78:0/53644740 conn(0x5565af732ec0 msgr2=0x5565af72dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcf13f7f740 1 -- 172.21.15.78:0/53644740 shutdown_connections 2024-04-25T01:45:33.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcf13f7f740 1 -- 172.21.15.78:0/53644740 wait complete. 2024-04-25T01:45:33.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.244+0000 7fcf13f7f740 20 client.15504 trim_cache size 0 max 16384 2024-04-25T01:45:33.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.264+0000 7fcf13f7f740 1 Processor -- start 2024-04-25T01:45:33.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf13f7f740 1 -- start start 2024-04-25T01:45:33.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf13f7f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af737850 0x5565af73f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf13f7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af738240 0x5565af749190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf13f7f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af73d3b0 0x5565af7332a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf13f7f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5565af74e0a0 con 0x5565af73d3b0 2024-04-25T01:45:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf13f7f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5565af74e770 con 0x5565af737850 2024-04-25T01:45:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf13f7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565af74ee40 con 0x5565af738240 2024-04-25T01:45:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf0259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af73d3b0 0x5565af7332a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf0259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af73d3b0 0x5565af7332a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60360/0 (socket says 172.21.15.78:60360) 2024-04-25T01:45:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf0259c640 1 -- 172.21.15.78:0/1578165693 learned_addr learned my addr 172.21.15.78:0/1578165693 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf0ad9d640 1 --2- 172.21.15.78:0/1578165693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af738240 0x5565af749190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf0259c640 1 --2- 172.21.15.78:0/1578165693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af737850 0x5565af73f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf0259c640 1 -- 172.21.15.78:0/1578165693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af738240 msgr2=0x5565af749190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf0259c640 1 --2- 172.21.15.78:0/1578165693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af738240 0x5565af749190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf0259c640 1 -- 172.21.15.78:0/1578165693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af737850 msgr2=0x5565af73f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf0259c640 1 --2- 172.21.15.78:0/1578165693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af737850 0x5565af73f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.265+0000 7fcf0259c640 1 -- 172.21.15.78:0/1578165693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565af732cf0 con 0x5565af73d3b0 2024-04-25T01:45:33.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:33 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/406455652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:45:33.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:33 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2095318100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:33.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:33 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1423047461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:33.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.266+0000 7fcf0259c640 1 --2- 172.21.15.78:0/1578165693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af73d3b0 0x5565af7332a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x5565af7389a0 tx=0x7fcef4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b978e4dc7ded55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.266+0000 7fcf00d99640 1 -- 172.21.15.78:0/1578165693 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef40217f0 con 0x5565af73d3b0 2024-04-25T01:45:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.266+0000 7fcf00d99640 1 -- 172.21.15.78:0/1578165693 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef4009a90 con 0x5565af73d3b0 2024-04-25T01:45:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.266+0000 7fcf00d99640 1 -- 172.21.15.78:0/1578165693 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef40051c0 con 0x5565af73d3b0 2024-04-25T01:45:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 -- 172.21.15.78:0/1578165693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af73d3b0 msgr2=0x5565af7332a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/1578165693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af73d3b0 0x5565af7332a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x5565af7389a0 tx=0x7fcef4001e70 comp rx=0 tx=0).stop 2024-04-25T01:45:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 -- 172.21.15.78:0/1578165693 shutdown_connections 2024-04-25T01:45:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/1578165693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af73d3b0 0x5565af7332a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/1578165693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af738240 0x5565af749190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/1578165693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af737850 0x5565af73f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 -- 172.21.15.78:0/1578165693 >> 172.21.15.78:0/1578165693 conn(0x5565af732ec0 msgr2=0x5565af738ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 -- 172.21.15.78:0/1578165693 shutdown_connections 2024-04-25T01:45:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 -- 172.21.15.78:0/1578165693 wait complete. 2024-04-25T01:45:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 -- start start 2024-04-25T01:45:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 Processor -- start 2024-04-25T01:45:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af737850 0x5565af747110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af738240 0x5565af7b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af73d3b0 0x5565af7b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5565af74fbe0 con 0x5565af73d3b0 2024-04-25T01:45:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5565af74f510 con 0x5565af738240 2024-04-25T01:45:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565af7502b0 con 0x5565af737850 2024-04-25T01:45:33.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf13f7f740 10 client.0 _ll_register_callbacks cb 0x7fcef40108e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf0259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af73d3b0 0x5565af7b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf0259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af73d3b0 0x5565af7b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60376/0 (socket says 172.21.15.78:60376) 2024-04-25T01:45:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.267+0000 7fcf0259c640 1 -- 172.21.15.78:0/340131959 learned_addr learned my addr 172.21.15.78:0/340131959 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcf0ad9d640 1 --2- 172.21.15.78:0/340131959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af738240 0x5565af7b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcf0259c640 1 --2- 172.21.15.78:0/340131959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af737850 0x5565af747110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcf0259c640 1 -- 172.21.15.78:0/340131959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af737850 msgr2=0x5565af747110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcf0259c640 1 --2- 172.21.15.78:0/340131959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af737850 0x5565af747110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcf0259c640 1 -- 172.21.15.78:0/340131959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af738240 msgr2=0x5565af7b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcf0259c640 1 --2- 172.21.15.78:0/340131959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af738240 0x5565af7b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcf0259c640 1 -- 172.21.15.78:0/340131959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565af7b3440 con 0x5565af73d3b0 2024-04-25T01:45:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcf0259c640 1 --2- 172.21.15.78:0/340131959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af73d3b0 0x5565af7b2dd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009310 tx=0x7fcef40098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abac7edae4c9fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcef27fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcef27fc640 1 -- 172.21.15.78:0/340131959 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef40217f0 con 0x5565af73d3b0 2024-04-25T01:45:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcef27fc640 1 -- 172.21.15.78:0/340131959 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fcef4021a40 con 0x5565af73d3b0 2024-04-25T01:45:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcf13f7f740 1 -- 172.21.15.78:0/340131959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef4009a90 con 0x5565af73d3b0 2024-04-25T01:45:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcef27fc640 1 -- 172.21.15.78:0/340131959 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcef4021c40 con 0x5565af73d3b0 2024-04-25T01:45:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcf13f7f740 10 client.15510 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcf13f7f740 1 -- 172.21.15.78:0/340131959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565af7b4740 con 0x5565af73d3b0 2024-04-25T01:45:33.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.268+0000 7fcef27fc640 1 -- 172.21.15.78:0/340131959 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef4009a90 con 0x5565af73d3b0 2024-04-25T01:45:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.269+0000 7fcef27fc640 1 -- 172.21.15.78:0/340131959 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcef401d070 con 0x5565af73d3b0 2024-04-25T01:45:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.269+0000 7fcf13f7f740 10 client.15510 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.269+0000 7fcf13f7f740 10 client.15510 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T01:45:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.269+0000 7fcf13f7f740 20 client.15510 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.269+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/340131959 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x5565af7b91b0 0x5565af7b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.269+0000 7fcf13f7f740 4 client.15510 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:45:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.269+0000 7fcf13f7f740 1 -- 172.21.15.78:0/340131959 --> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] -- command(tid 0: {"prefix": "status"}) -- 0x5565af69e790 con 0x5565af7b91b0 2024-04-25T01:45:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.269+0000 7fcf0ad9d640 1 --2- 172.21.15.78:0/340131959 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x5565af7b91b0 0x5565af7b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.269+0000 7fcf0ad9d640 1 --2- 172.21.15.78:0/340131959 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x5565af7b91b0 0x5565af7b7ec0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9bbef519bcd9389 server_cookie=2ad61f08f3c742d7 in_seq=0 out_seq=0 2024-04-25T01:45:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.269+0000 7fcef27fc640 10 client.15510 ms_handle_connect on v2:172.21.15.152:6836/4112159743 2024-04-25T01:45:33.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcef27fc640 1 -- 172.21.15.78:0/340131959 <== mds.1 v2:172.21.15.152:6836/4112159743 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5565af69e790 con 0x5565af7b91b0 2024-04-25T01:45:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcef27fc640 10 client.15510 handle_command_reply: tid=0 2024-04-25T01:45:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcf13f7f740 1 client.15510 shutdown 2024-04-25T01:45:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcf13f7f740 1 -- 172.21.15.78:0/340131959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af73d3b0 msgr2=0x5565af7b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/340131959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af73d3b0 0x5565af7b2dd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009310 tx=0x7fcef40098d0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcf13f7f740 1 -- 172.21.15.78:0/340131959 shutdown_connections 2024-04-25T01:45:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/340131959 >> [v2:172.21.15.152:6836/4112159743,v1:172.21.15.152:6837/4112159743] conn(0x5565af7b91b0 0x5565af7b7ec0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/340131959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5565af73d3b0 0x5565af7b2dd0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcef27fc640 0 client.15510 ms_handle_reset on v2:172.21.15.152:6836/4112159743 2024-04-25T01:45:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/340131959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565af738240 0x5565af7b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcf13f7f740 1 --2- 172.21.15.78:0/340131959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565af737850 0x5565af747110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcf13f7f740 1 -- 172.21.15.78:0/340131959 >> 172.21.15.78:0/340131959 conn(0x5565af732ec0 msgr2=0x5565af7332a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcf13f7f740 1 -- 172.21.15.78:0/340131959 shutdown_connections 2024-04-25T01:45:33.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcf13f7f740 1 -- 172.21.15.78:0/340131959 wait complete. 2024-04-25T01:45:33.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.270+0000 7fcf13f7f740 20 client.15510 trim_cache size 0 max 16384 2024-04-25T01:45:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:45:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "df36bea8-02a3-11ef-bc93-c7b262605968", 2024-04-25T01:45:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 1, 2024-04-25T01:45:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24455, 2024-04-25T01:45:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T01:45:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T01:45:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T01:45:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 112.27905178100001, 2024-04-25T01:45:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-04-25T01:45:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 94, 2024-04-25T01:45:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T01:45:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 152.64918164900001 2024-04-25T01:45:33.274 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:45:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.271+0000 7fcf11190640 1 -- 172.21.15.78:0/708687126 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fcef80a2b90 msgr2=0x7fcef80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.271+0000 7fcf11190640 1 --2- 172.21.15.78:0/708687126 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fcef80a2b90 0x7fcef80a5050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcf04082d10 tx=0x7fcf0403d040 comp rx=0 tx=0).stop 2024-04-25T01:45:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.271+0000 7fcf11190640 1 -- 172.21.15.78:0/708687126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0c102230 msgr2=0x7fcf0c19edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.271+0000 7fcf11190640 1 --2- 172.21.15.78:0/708687126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0c102230 0x7fcf0c19edc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcf04031ae0 tx=0x7fcf04031ee0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.271+0000 7fcf11190640 1 -- 172.21.15.78:0/708687126 shutdown_connections 2024-04-25T01:45:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.271+0000 7fcf11190640 1 --2- 172.21.15.78:0/708687126 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fcef80a2b90 0x7fcef80a5050 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.271+0000 7fcf11190640 1 --2- 172.21.15.78:0/708687126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcf0c102230 0x7fcf0c19edc0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.271+0000 7fcf11190640 1 --2- 172.21.15.78:0/708687126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf0c101870 0x7fcf0c19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.271+0000 7fcf11190640 1 --2- 172.21.15.78:0/708687126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf0c100680 0x7fcf0c069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.271+0000 7fcf11190640 1 -- 172.21.15.78:0/708687126 >> 172.21.15.78:0/708687126 conn(0x7fcf0c0fbbf0 msgr2=0x7fcf0c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.271+0000 7fcf11190640 1 -- 172.21.15.78:0/708687126 shutdown_connections 2024-04-25T01:45:33.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.271+0000 7fcf11190640 1 -- 172.21.15.78:0/708687126 wait complete. 2024-04-25T01:45:33.277 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T01:45:33.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:33 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/406455652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:45:33.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:33 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2095318100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:33.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:33 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1423047461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:33.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:33 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/406455652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:45:33.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:33 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2095318100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:45:33.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:33 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1423047461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:45:33.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.385+0000 7fd6d36d6640 1 -- 172.21.15.78:0/3446746782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6cc102980 msgr2=0x7fd6cc102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.385+0000 7fd6d36d6640 1 --2- 172.21.15.78:0/3446746782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6cc102980 0x7fd6cc102e00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009900 tx=0x7fd6c002a410 comp rx=0 tx=0).stop 2024-04-25T01:45:33.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.385+0000 7fd6d36d6640 1 -- 172.21.15.78:0/3446746782 shutdown_connections 2024-04-25T01:45:33.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.385+0000 7fd6d36d6640 1 --2- 172.21.15.78:0/3446746782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6cc103340 0x7fd6cc109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.385+0000 7fd6d36d6640 1 --2- 172.21.15.78:0/3446746782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6cc102980 0x7fd6cc102e00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.385+0000 7fd6d36d6640 1 --2- 172.21.15.78:0/3446746782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6cc101790 0x7fd6cc101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.385+0000 7fd6d36d6640 1 -- 172.21.15.78:0/3446746782 >> 172.21.15.78:0/3446746782 conn(0x7fd6cc0fcd20 msgr2=0x7fd6cc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.386+0000 7fd6d36d6640 1 -- 172.21.15.78:0/3446746782 shutdown_connections 2024-04-25T01:45:33.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.386+0000 7fd6d36d6640 1 -- 172.21.15.78:0/3446746782 wait complete. 2024-04-25T01:45:33.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.386+0000 7fd6d36d6640 1 Processor -- start 2024-04-25T01:45:33.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.386+0000 7fd6d36d6640 1 -- start start 2024-04-25T01:45:33.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.386+0000 7fd6d36d6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6cc101790 0x7fd6cc197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.386+0000 7fd6d36d6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6cc102980 0x7fd6cc1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.386+0000 7fd6d36d6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6cc103340 0x7fd6cc19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.386+0000 7fd6d36d6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc10d480 con 0x7fd6cc101790 2024-04-25T01:45:33.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.386+0000 7fd6d36d6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc10d300 con 0x7fd6cc103340 2024-04-25T01:45:33.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.386+0000 7fd6d36d6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc10d600 con 0x7fd6cc102980 2024-04-25T01:45:33.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.386+0000 7fd6d1472640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6cc101790 0x7fd6cc197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.386+0000 7fd6d1472640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6cc101790 0x7fd6cc197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60402/0 (socket says 172.21.15.78:60402) 2024-04-25T01:45:33.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.386+0000 7fd6d1472640 1 -- 172.21.15.78:0/3539308180 learned_addr learned my addr 172.21.15.78:0/3539308180 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.387+0000 7fd6d0c71640 1 --2- 172.21.15.78:0/3539308180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6cc102980 0x7fd6cc1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.387+0000 7fd6d1472640 1 -- 172.21.15.78:0/3539308180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6cc102980 msgr2=0x7fd6cc1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.387+0000 7fd6d1472640 1 --2- 172.21.15.78:0/3539308180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6cc102980 0x7fd6cc1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.387+0000 7fd6d1472640 1 -- 172.21.15.78:0/3539308180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6cc103340 msgr2=0x7fd6cc19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.387+0000 7fd6d1472640 1 --2- 172.21.15.78:0/3539308180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6cc103340 0x7fd6cc19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.387+0000 7fd6d1472640 1 -- 172.21.15.78:0/3539308180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6cc19f7a0 con 0x7fd6cc101790 2024-04-25T01:45:33.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.387+0000 7fd6d1472640 1 --2- 172.21.15.78:0/3539308180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6cc101790 0x7fd6cc197b80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400b450 tx=0x7fd6c400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.387+0000 7fd6ca7fc640 1 -- 172.21.15.78:0/3539308180 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4015020 con 0x7fd6cc101790 2024-04-25T01:45:33.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.387+0000 7fd6d36d6640 1 -- 172.21.15.78:0/3539308180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6cc19fa90 con 0x7fd6cc101790 2024-04-25T01:45:33.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.387+0000 7fd6ca7fc640 1 -- 172.21.15.78:0/3539308180 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6c4011d70 con 0x7fd6cc101790 2024-04-25T01:45:33.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.387+0000 7fd6ca7fc640 1 -- 172.21.15.78:0/3539308180 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c400f340 con 0x7fd6cc101790 2024-04-25T01:45:33.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.387+0000 7fd6d36d6640 1 -- 172.21.15.78:0/3539308180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6cc1a0000 con 0x7fd6cc101790 2024-04-25T01:45:33.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.389+0000 7fd6ca7fc640 1 -- 172.21.15.78:0/3539308180 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd6c4018020 con 0x7fd6cc101790 2024-04-25T01:45:33.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.389+0000 7fd6d63e2740 1 Processor -- start 2024-04-25T01:45:33.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.389+0000 7fd6d63e2740 1 -- start start 2024-04-25T01:45:33.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d63e2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17c440 0x55945c17c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d63e2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c17ce30 0x55945c181930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d63e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c181fa0 0x55945c184430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d63e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55945c0dff00 con 0x55945c181fa0 2024-04-25T01:45:33.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d63e2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55945c088750 con 0x55945c17ce30 2024-04-25T01:45:33.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d63e2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55945c0dbec0 con 0x55945c17c440 2024-04-25T01:45:33.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d0c71640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c181fa0 0x55945c184430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d0c71640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c181fa0 0x55945c184430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60404/0 (socket says 172.21.15.78:60404) 2024-04-25T01:45:33.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d0c71640 1 -- 172.21.15.78:0/2803401229 learned_addr learned my addr 172.21.15.78:0/2803401229 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6ca7fc640 1 --2- 172.21.15.78:0/3539308180 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd6b40a2b90 0x7fd6b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d1472640 1 --2- 172.21.15.78:0/2803401229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c17ce30 0x55945c181930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d1c73640 1 --2- 172.21.15.78:0/2803401229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17c440 0x55945c17c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d0c71640 1 -- 172.21.15.78:0/2803401229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17c440 msgr2=0x55945c17c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d0c71640 1 --2- 172.21.15.78:0/2803401229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17c440 0x55945c17c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d0c71640 1 -- 172.21.15.78:0/2803401229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c17ce30 msgr2=0x55945c181930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6ca7fc640 1 -- 172.21.15.78:0/3539308180 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fd6c40cef30 con 0x7fd6cc101790 2024-04-25T01:45:33.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d0c71640 1 --2- 172.21.15.78:0/2803401229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c17ce30 0x55945c181930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d0c71640 1 -- 172.21.15.78:0/2803401229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945bdf42d0 con 0x55945c181fa0 2024-04-25T01:45:33.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d1c73640 1 --2- 172.21.15.78:0/2803401229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17c440 0x55945c17c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:33.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d1c73640 1 --2- 172.21.15.78:0/3539308180 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd6b40a2b90 0x7fd6b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.390+0000 7fd6d0c71640 1 --2- 172.21.15.78:0/2803401229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c181fa0 0x55945c184430 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd6c002a930 tx=0x7fd6c0031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98bb18b763bd73c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.391+0000 7fd6b37fe640 1 -- 172.21.15.78:0/2803401229 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c003f070 con 0x55945c181fa0 2024-04-25T01:45:33.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.391+0000 7fd6b37fe640 1 -- 172.21.15.78:0/2803401229 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6c0031a10 con 0x55945c181fa0 2024-04-25T01:45:33.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.391+0000 7fd6b37fe640 1 -- 172.21.15.78:0/2803401229 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0035070 con 0x55945c181fa0 2024-04-25T01:45:33.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.391+0000 7fd6d1c73640 1 --2- 172.21.15.78:0/3539308180 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd6b40a2b90 0x7fd6b40a5050 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55945c17d690 tx=0x7fd6bc004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.391+0000 7fd6d63e2740 1 -- 172.21.15.78:0/2803401229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c181fa0 msgr2=0x55945c184430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.391+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/2803401229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c181fa0 0x55945c184430 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd6c002a930 tx=0x7fd6c0031760 comp rx=0 tx=0).stop 2024-04-25T01:45:33.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.391+0000 7fd6d63e2740 1 -- 172.21.15.78:0/2803401229 shutdown_connections 2024-04-25T01:45:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.391+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/2803401229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c181fa0 0x55945c184430 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.391+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/2803401229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c17ce30 0x55945c181930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.391+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/2803401229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17c440 0x55945c17c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.391+0000 7fd6d63e2740 1 -- 172.21.15.78:0/2803401229 >> 172.21.15.78:0/2803401229 conn(0x55945c177ab0 msgr2=0x55945c177eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.391+0000 7fd6d63e2740 1 -- 172.21.15.78:0/2803401229 shutdown_connections 2024-04-25T01:45:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.391+0000 7fd6d63e2740 1 -- 172.21.15.78:0/2803401229 wait complete. 2024-04-25T01:45:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d63e2740 1 -- start start 2024-04-25T01:45:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d63e2740 1 Processor -- start 2024-04-25T01:45:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d63e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c17c440 0x55945c186780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d63e2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17ce30 0x55945c186cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d63e2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c181fa0 0x55945c18dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d63e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55945c184ef0 con 0x55945c17c440 2024-04-25T01:45:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d63e2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55945c09bcf0 con 0x55945c181fa0 2024-04-25T01:45:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d63e2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55945c185070 con 0x55945c17ce30 2024-04-25T01:45:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d63e2740 10 client.0 _ll_register_callbacks cb 0x55945c09b6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d1472640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c17c440 0x55945c186780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d1472640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c17c440 0x55945c186780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60420/0 (socket says 172.21.15.78:60420) 2024-04-25T01:45:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d1472640 1 -- 172.21.15.78:0/2844221634 learned_addr learned my addr 172.21.15.78:0/2844221634 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d1c73640 1 --2- 172.21.15.78:0/2844221634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c181fa0 0x55945c18dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d0c71640 1 --2- 172.21.15.78:0/2844221634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17ce30 0x55945c186cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d1472640 1 -- 172.21.15.78:0/2844221634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17ce30 msgr2=0x55945c186cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d1472640 1 --2- 172.21.15.78:0/2844221634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17ce30 0x55945c186cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d1472640 1 -- 172.21.15.78:0/2844221634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c181fa0 msgr2=0x55945c18dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d1472640 1 --2- 172.21.15.78:0/2844221634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c181fa0 0x55945c18dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.392+0000 7fd6d1472640 1 -- 172.21.15.78:0/2844221634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945bebe6b0 con 0x55945c17c440 2024-04-25T01:45:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6d1472640 1 --2- 172.21.15.78:0/2844221634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c17c440 0x55945c186780 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd6c409b0b0 tx=0x7fd6c409b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dae72e4f93a1df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6b0ff9640 1 -- 172.21.15.78:0/2844221634 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c409b3b0 con 0x55945c17c440 2024-04-25T01:45:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6b0ff9640 1 -- 172.21.15.78:0/2844221634 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fd6c4010570 con 0x55945c17c440 2024-04-25T01:45:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6d63e2740 1 -- 172.21.15.78:0/2844221634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55945c0ed440 con 0x55945c17c440 2024-04-25T01:45:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6b0ff9640 1 -- 172.21.15.78:0/2844221634 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd6c409f760 con 0x55945c17c440 2024-04-25T01:45:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6d63e2740 10 client.15522 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6d63e2740 1 -- 172.21.15.78:0/2844221634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55945c190cc0 con 0x55945c17c440 2024-04-25T01:45:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6b0ff9640 1 -- 172.21.15.78:0/2844221634 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55945c0ed440 con 0x55945c17c440 2024-04-25T01:45:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6b0ff9640 1 -- 172.21.15.78:0/2844221634 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd6c409a800 con 0x55945c17c440 2024-04-25T01:45:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6d63e2740 10 client.15522 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6d63e2740 10 client.15522 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T01:45:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6d63e2740 20 client.15522 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/2844221634 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x55945c193330 0x55945c1947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6d63e2740 4 client.15522 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:45:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6d63e2740 1 -- 172.21.15.78:0/2844221634 --> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55945c0e7790 con 0x55945c193330 2024-04-25T01:45:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.393+0000 7fd6d0c71640 1 --2- 172.21.15.78:0/2844221634 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x55945c193330 0x55945c1947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.394+0000 7fd6b0ff9640 10 client.15522 ms_handle_connect on v2:172.21.15.152:6838/1969344843 2024-04-25T01:45:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.394+0000 7fd6d0c71640 1 --2- 172.21.15.78:0/2844221634 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x55945c193330 0x55945c1947a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac4a16c6b714e6cb server_cookie=cc2173919fd23e6f in_seq=0 out_seq=0 2024-04-25T01:45:33.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6b0ff9640 1 -- 172.21.15.78:0/2844221634 <== mds.0 v2:172.21.15.152:6838/1969344843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55945c0e7790 con 0x55945c193330 2024-04-25T01:45:33.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6b0ff9640 10 client.15522 handle_command_reply: tid=0 2024-04-25T01:45:33.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6d63e2740 1 client.15522 shutdown 2024-04-25T01:45:33.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6d63e2740 1 -- 172.21.15.78:0/2844221634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c17c440 msgr2=0x55945c186780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/2844221634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c17c440 0x55945c186780 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd6c409b0b0 tx=0x7fd6c409b290 comp rx=0 tx=0).stop 2024-04-25T01:45:33.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6d63e2740 1 -- 172.21.15.78:0/2844221634 shutdown_connections 2024-04-25T01:45:33.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/2844221634 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x55945c193330 0x55945c1947a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/2844221634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c181fa0 0x55945c18dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/2844221634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17ce30 0x55945c186cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6b0ff9640 0 client.15522 ms_handle_reset on v2:172.21.15.152:6838/1969344843 2024-04-25T01:45:33.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/2844221634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c17c440 0x55945c186780 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6d63e2740 1 -- 172.21.15.78:0/2844221634 >> 172.21.15.78:0/2844221634 conn(0x55945c177ab0 msgr2=0x55945c176ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6d63e2740 1 -- 172.21.15.78:0/2844221634 shutdown_connections 2024-04-25T01:45:33.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6d63e2740 1 -- 172.21.15.78:0/2844221634 wait complete. 2024-04-25T01:45:33.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.395+0000 7fd6d63e2740 20 client.15522 trim_cache size 0 max 16384 2024-04-25T01:45:33.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d63e2740 1 Processor -- start 2024-04-25T01:45:33.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d63e2740 1 -- start start 2024-04-25T01:45:33.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d63e2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c17c440 0x55945c172690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d63e2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17ce30 0x55945c18dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d63e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c181fa0 0x55945c17e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d63e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55945c192ae0 con 0x55945c181fa0 2024-04-25T01:45:33.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d63e2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55945c1931b0 con 0x55945c17c440 2024-04-25T01:45:33.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d63e2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55945c193880 con 0x55945c17ce30 2024-04-25T01:45:33.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d1c73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c181fa0 0x55945c17e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d1c73640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c181fa0 0x55945c17e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60432/0 (socket says 172.21.15.78:60432) 2024-04-25T01:45:33.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d1c73640 1 -- 172.21.15.78:0/1854173066 learned_addr learned my addr 172.21.15.78:0/1854173066 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d1472640 1 --2- 172.21.15.78:0/1854173066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c17c440 0x55945c172690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d1c73640 1 -- 172.21.15.78:0/1854173066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17ce30 msgr2=0x55945c18dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d1c73640 1 --2- 172.21.15.78:0/1854173066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17ce30 0x55945c18dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d1c73640 1 -- 172.21.15.78:0/1854173066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c17c440 msgr2=0x55945c172690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d1c73640 1 --2- 172.21.15.78:0/1854173066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c17c440 0x55945c172690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d1c73640 1 -- 172.21.15.78:0/1854173066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55945c17d210 con 0x55945c181fa0 2024-04-25T01:45:33.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d1472640 1 --2- 172.21.15.78:0/1854173066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c17c440 0x55945c172690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:33.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6d1c73640 1 --2- 172.21.15.78:0/1854173066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c181fa0 0x55945c17e190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc005f20 tx=0x7fd6bc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6f9286063b4a87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.413+0000 7fd6b2ffd640 1 -- 172.21.15.78:0/1854173066 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc01c6f0 con 0x55945c181fa0 2024-04-25T01:45:33.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.414+0000 7fd6b2ffd640 1 -- 172.21.15.78:0/1854173066 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6bc01cd70 con 0x55945c181fa0 2024-04-25T01:45:33.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.414+0000 7fd6b2ffd640 1 -- 172.21.15.78:0/1854173066 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc01be50 con 0x55945c181fa0 2024-04-25T01:45:33.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.414+0000 7fd6d63e2740 1 -- 172.21.15.78:0/1854173066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c181fa0 msgr2=0x55945c17e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.414+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/1854173066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c181fa0 0x55945c17e190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc005f20 tx=0x7fd6bc010000 comp rx=0 tx=0).stop 2024-04-25T01:45:33.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.414+0000 7fd6d63e2740 1 -- 172.21.15.78:0/1854173066 shutdown_connections 2024-04-25T01:45:33.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.414+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/1854173066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c181fa0 0x55945c17e190 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.414+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/1854173066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17ce30 0x55945c18dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.414+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/1854173066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c17c440 0x55945c172690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.414+0000 7fd6d63e2740 1 -- 172.21.15.78:0/1854173066 >> 172.21.15.78:0/1854173066 conn(0x55945c177ab0 msgr2=0x55945c177e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.414+0000 7fd6d63e2740 1 -- 172.21.15.78:0/1854173066 shutdown_connections 2024-04-25T01:45:33.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.414+0000 7fd6d63e2740 1 -- 172.21.15.78:0/1854173066 wait complete. 2024-04-25T01:45:33.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.414+0000 7fd6d63e2740 1 -- start start 2024-04-25T01:45:33.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.414+0000 7fd6d63e2740 1 Processor -- start 2024-04-25T01:45:33.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d63e2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c17c440 0x55945c18bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d63e2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17ce30 0x55945c1f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d63e2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c181fa0 0x55945c1f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d63e2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55945c194620 con 0x55945c17c440 2024-04-25T01:45:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d63e2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55945c193f50 con 0x55945c181fa0 2024-04-25T01:45:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d63e2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55945c194cf0 con 0x55945c17ce30 2024-04-25T01:45:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d63e2740 10 client.0 _ll_register_callbacks cb 0x7fd6c4097740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:45:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d1472640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c17c440 0x55945c18bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d1472640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c17c440 0x55945c18bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60442/0 (socket says 172.21.15.78:60442) 2024-04-25T01:45:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d1472640 1 -- 172.21.15.78:0/396591458 learned_addr learned my addr 172.21.15.78:0/396591458 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d1c73640 1 --2- 172.21.15.78:0/396591458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c181fa0 0x55945c1f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d0c71640 1 --2- 172.21.15.78:0/396591458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17ce30 0x55945c1f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d1472640 1 -- 172.21.15.78:0/396591458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17ce30 msgr2=0x55945c1f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d1472640 1 --2- 172.21.15.78:0/396591458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17ce30 0x55945c1f52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d1472640 1 -- 172.21.15.78:0/396591458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c181fa0 msgr2=0x55945c1f78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:45:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d1472640 1 --2- 172.21.15.78:0/396591458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c181fa0 0x55945c1f78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d1472640 1 -- 172.21.15.78:0/396591458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945c1f80e0 con 0x55945c17c440 2024-04-25T01:45:33.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d1c73640 1 --2- 172.21.15.78:0/396591458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c181fa0 0x55945c1f78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:45:33.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d1472640 1 --2- 172.21.15.78:0/396591458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c17c440 0x55945c18bc10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd6c409b340 tx=0x7fd6c40998c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9993f0a018e7d753 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:33.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-04-25T01:45:33.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6b0ff9640 1 -- 172.21.15.78:0/396591458 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c40c9070 con 0x55945c17c440 2024-04-25T01:45:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6b0ff9640 1 -- 172.21.15.78:0/396591458 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fd6c409a800 con 0x55945c17c440 2024-04-25T01:45:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.415+0000 7fd6d63e2740 1 -- 172.21.15.78:0/396591458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6bc01cd70 con 0x55945c17c440 2024-04-25T01:45:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.416+0000 7fd6b0ff9640 1 -- 172.21.15.78:0/396591458 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd6c409ad10 con 0x55945c17c440 2024-04-25T01:45:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.416+0000 7fd6d63e2740 10 client.15528 fetch_fsmap learned FSMap version 28 2024-04-25T01:45:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.416+0000 7fd6d63e2740 1 -- 172.21.15.78:0/396591458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55945c1f9350 con 0x55945c17c440 2024-04-25T01:45:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.416+0000 7fd6b0ff9640 1 -- 172.21.15.78:0/396591458 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6bc01cd70 con 0x55945c17c440 2024-04-25T01:45:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.416+0000 7fd6b0ff9640 1 -- 172.21.15.78:0/396591458 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd6c4066210 con 0x55945c17c440 2024-04-25T01:45:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.416+0000 7fd6d63e2740 10 client.15528 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T01:45:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.416+0000 7fd6d63e2740 10 client.15528 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T01:45:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.416+0000 7fd6d63e2740 20 client.15528 populate_metadata read hostname 'smithi078' 2024-04-25T01:45:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.416+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/396591458 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x55945c1fdf40 0x55945c1fcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.416+0000 7fd6d63e2740 4 client.15528 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:45:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.416+0000 7fd6d63e2740 1 -- 172.21.15.78:0/396591458 --> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] -- command(tid 0: {"prefix": "status"}) -- 0x55945c065b80 con 0x55945c1fdf40 2024-04-25T01:45:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.416+0000 7fd6d0c71640 1 --2- 172.21.15.78:0/396591458 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x55945c1fdf40 0x55945c1fcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.416+0000 7fd6b0ff9640 10 client.15528 ms_handle_connect on v2:172.21.15.152:6838/1969344843 2024-04-25T01:45:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.416+0000 7fd6d0c71640 1 --2- 172.21.15.78:0/396591458 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x55945c1fdf40 0x55945c1fcc50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0dab52d89f5016 server_cookie=acd79ab9ca82649a in_seq=0 out_seq=0 2024-04-25T01:45:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6b0ff9640 1 -- 172.21.15.78:0/396591458 <== mds.0 v2:172.21.15.152:6838/1969344843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55945c065b80 con 0x55945c1fdf40 2024-04-25T01:45:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6b0ff9640 10 client.15528 handle_command_reply: tid=0 2024-04-25T01:45:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d63e2740 1 client.15528 shutdown 2024-04-25T01:45:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d63e2740 1 -- 172.21.15.78:0/396591458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c17c440 msgr2=0x55945c18bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/396591458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c17c440 0x55945c18bc10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd6c409b340 tx=0x7fd6c40998c0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d63e2740 1 -- 172.21.15.78:0/396591458 shutdown_connections 2024-04-25T01:45:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/396591458 >> [v2:172.21.15.152:6838/1969344843,v1:172.21.15.152:6839/1969344843] conn(0x55945c1fdf40 0x55945c1fcc50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/396591458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55945c181fa0 0x55945c1f78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/396591458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55945c17ce30 0x55945c1f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d63e2740 1 --2- 172.21.15.78:0/396591458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55945c17c440 0x55945c18bc10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6b0ff9640 0 client.15528 ms_handle_reset on v2:172.21.15.152:6838/1969344843 2024-04-25T01:45:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d63e2740 1 -- 172.21.15.78:0/396591458 >> 172.21.15.78:0/396591458 conn(0x55945c177ab0 msgr2=0x55945c172690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d63e2740 1 -- 172.21.15.78:0/396591458 shutdown_connections 2024-04-25T01:45:33.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d63e2740 1 -- 172.21.15.78:0/396591458 wait complete. 2024-04-25T01:45:33.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d63e2740 20 client.15528 trim_cache size 0 max 16384 2024-04-25T01:45:33.421 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:45:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "df36bea8-02a3-11ef-bc93-c7b262605968", 2024-04-25T01:45:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-04-25T01:45:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24491, 2024-04-25T01:45:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-04-25T01:45:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-04-25T01:45:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-04-25T01:45:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 135.38935876100001, 2024-04-25T01:45:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-04-25T01:45:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 94, 2024-04-25T01:45:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T01:45:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 141.48424761000001 2024-04-25T01:45:33.422 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:45:33.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d36d6640 1 -- 172.21.15.78:0/3539308180 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd6b40a2b90 msgr2=0x7fd6b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d36d6640 1 --2- 172.21.15.78:0/3539308180 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd6b40a2b90 0x7fd6b40a5050 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55945c17d690 tx=0x7fd6bc004a40 comp rx=0 tx=0).stop 2024-04-25T01:45:33.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d36d6640 1 -- 172.21.15.78:0/3539308180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6cc101790 msgr2=0x7fd6cc197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:33.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.417+0000 7fd6d36d6640 1 --2- 172.21.15.78:0/3539308180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6cc101790 0x7fd6cc197b80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400b450 tx=0x7fd6c400b910 comp rx=0 tx=0).stop 2024-04-25T01:45:33.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.418+0000 7fd6d36d6640 1 -- 172.21.15.78:0/3539308180 shutdown_connections 2024-04-25T01:45:33.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.418+0000 7fd6d36d6640 1 --2- 172.21.15.78:0/3539308180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6cc103340 0x7fd6cc19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.418+0000 7fd6d36d6640 1 --2- 172.21.15.78:0/3539308180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6cc102980 0x7fd6cc1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.418+0000 7fd6d36d6640 1 --2- 172.21.15.78:0/3539308180 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fd6b40a2b90 0x7fd6b40a5050 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.418+0000 7fd6d36d6640 1 --2- 172.21.15.78:0/3539308180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6cc101790 0x7fd6cc197b80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:33.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.418+0000 7fd6d36d6640 1 -- 172.21.15.78:0/3539308180 >> 172.21.15.78:0/3539308180 conn(0x7fd6cc0fcd20 msgr2=0x7fd6cc105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:33.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.418+0000 7fd6d36d6640 1 -- 172.21.15.78:0/3539308180 shutdown_connections 2024-04-25T01:45:33.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:33.418+0000 7fd6d36d6640 1 -- 172.21.15.78:0/3539308180 wait complete. 2024-04-25T01:45:33.425 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T01:45:33.435 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T01:45:33.435 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T01:45:33.435 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T01:45:33.435 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T01:45:33.447 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T01:45:33.448 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T01:45:33.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:45:33.448 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-04-25T01:45:33.463 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T01:45:33.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:45:33.463 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:45:33.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:45:33.528 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-04-25T01:45:33.626 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T01:45:33.626 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:45:33.626 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T01:45:33.626 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T01:45:33.863 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:45:34.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:34 smithi078 ceph-mon[29469]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 5.2 KiB/s, 1 objects/s recovering 2024-04-25T01:45:34.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:34 smithi152 ceph-mon[32465]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 5.2 KiB/s, 1 objects/s recovering 2024-04-25T01:45:34.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:34 smithi196 ceph-mon[32978]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 5.2 KiB/s, 1 objects/s recovering 2024-04-25T01:45:35.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.596+0000 7ff58f027700 1 -- 172.21.15.78:0/2968900601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff588101130 msgr2=0x7ff5881015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:35.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.596+0000 7ff58f027700 1 --2- 172.21.15.78:0/2968900601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff588101130 0x7ff5881015b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff574009990 tx=0x7ff574009ca0 comp rx=0 tx=0).stop 2024-04-25T01:45:35.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.596+0000 7ff58f027700 1 -- 172.21.15.78:0/2968900601 shutdown_connections 2024-04-25T01:45:35.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.596+0000 7ff58f027700 1 --2- 172.21.15.78:0/2968900601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5881071a0 0x7ff588107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:35.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.596+0000 7ff58f027700 1 --2- 172.21.15.78:0/2968900601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff588101af0 0x7ff58810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:35.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.596+0000 7ff58f027700 1 --2- 172.21.15.78:0/2968900601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff588101130 0x7ff5881015b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:35.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.596+0000 7ff58f027700 1 -- 172.21.15.78:0/2968900601 >> 172.21.15.78:0/2968900601 conn(0x7ff5880780e0 msgr2=0x7ff5880784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:35.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.596+0000 7ff58f027700 1 -- 172.21.15.78:0/2968900601 shutdown_connections 2024-04-25T01:45:35.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.596+0000 7ff58f027700 1 -- 172.21.15.78:0/2968900601 wait complete. 2024-04-25T01:45:35.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.596+0000 7ff58f027700 1 Processor -- start 2024-04-25T01:45:35.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.596+0000 7ff58f027700 1 -- start start 2024-04-25T01:45:35.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58f027700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff588101130 0x7ff58819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:35.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58f027700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff588101af0 0x7ff58819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:35.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58f027700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5881071a0 0x7ff588197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:35.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58f027700 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff588111430 con 0x7ff5881071a0 2024-04-25T01:45:35.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58f027700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff5881112b0 con 0x7ff588101af0 2024-04-25T01:45:35.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58f027700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5881115b0 con 0x7ff588101130 2024-04-25T01:45:35.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58d5c4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5881071a0 0x7ff588197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:35.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58d5c4700 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5881071a0 0x7ff588197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60458/0 (socket says 172.21.15.78:60458) 2024-04-25T01:45:35.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58d5c4700 1 -- 172.21.15.78:0/1968849139 learned_addr learned my addr 172.21.15.78:0/1968849139 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:35.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58cdc3700 1 --2- 172.21.15.78:0/1968849139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff588101130 0x7ff58819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:35.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff587fff700 1 --2- 172.21.15.78:0/1968849139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff588101af0 0x7ff58819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:35.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58d5c4700 1 -- 172.21.15.78:0/1968849139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff588101130 msgr2=0x7ff58819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:35.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58d5c4700 1 --2- 172.21.15.78:0/1968849139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff588101130 0x7ff58819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:35.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58d5c4700 1 -- 172.21.15.78:0/1968849139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff588101af0 msgr2=0x7ff58819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:35.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58d5c4700 1 --2- 172.21.15.78:0/1968849139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff588101af0 0x7ff58819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:35.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58d5c4700 1 -- 172.21.15.78:0/1968849139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff588198050 con 0x7ff5881071a0 2024-04-25T01:45:35.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58d5c4700 1 --2- 172.21.15.78:0/1968849139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5881071a0 0x7ff588197770 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff580007ee0 tx=0x7ff58000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:35.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff585ffb700 1 -- 172.21.15.78:0/1968849139 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58000ddd0 con 0x7ff5881071a0 2024-04-25T01:45:35.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff585ffb700 1 -- 172.21.15.78:0/1968849139 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff580004020 con 0x7ff5881071a0 2024-04-25T01:45:35.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff58f027700 1 -- 172.21.15.78:0/1968849139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff588198340 con 0x7ff5881071a0 2024-04-25T01:45:35.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.597+0000 7ff585ffb700 1 -- 172.21.15.78:0/1968849139 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580015660 con 0x7ff5881071a0 2024-04-25T01:45:35.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.598+0000 7ff58f027700 1 -- 172.21.15.78:0/1968849139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5881a4620 con 0x7ff5881071a0 2024-04-25T01:45:35.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.599+0000 7ff58f027700 1 -- 172.21.15.78:0/1968849139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff588102850 con 0x7ff5881071a0 2024-04-25T01:45:35.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.600+0000 7ff585ffb700 1 -- 172.21.15.78:0/1968849139 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff5800041d0 con 0x7ff5881071a0 2024-04-25T01:45:35.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.600+0000 7ff585ffb700 1 --2- 172.21.15.78:0/1968849139 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff5780a2c20 0x7ff5780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:35.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.600+0000 7ff58cdc3700 1 --2- 172.21.15.78:0/1968849139 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff5780a2c20 0x7ff5780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:35.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.601+0000 7ff585ffb700 1 -- 172.21.15.78:0/1968849139 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7ff5800cee90 con 0x7ff5881071a0 2024-04-25T01:45:35.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.601+0000 7ff58cdc3700 1 --2- 172.21.15.78:0/1968849139 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff5780a2c20 0x7ff5780a50e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009be0 tx=0x7ff57c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:35.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.605+0000 7ff585ffb700 1 -- 172.21.15.78:0/1968849139 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff580010070 con 0x7ff5881071a0 2024-04-25T01:45:35.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.845+0000 7ff58f027700 1 -- 172.21.15.78:0/1968849139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff588198570 con 0x7ff5881071a0 2024-04-25T01:45:35.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.845+0000 7ff585ffb700 1 -- 172.21.15.78:0/1968849139 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7ff58009ba10 con 0x7ff5881071a0 2024-04-25T01:45:35.840 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T01:45:35.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.848+0000 7ff58f027700 1 -- 172.21.15.78:0/1968849139 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff5780a2c20 msgr2=0x7ff5780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:35.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.848+0000 7ff58f027700 1 --2- 172.21.15.78:0/1968849139 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff5780a2c20 0x7ff5780a50e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009be0 tx=0x7ff57c009260 comp rx=0 tx=0).stop 2024-04-25T01:45:35.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.848+0000 7ff58f027700 1 -- 172.21.15.78:0/1968849139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5881071a0 msgr2=0x7ff588197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:35.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.848+0000 7ff58f027700 1 --2- 172.21.15.78:0/1968849139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5881071a0 0x7ff588197770 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff580007ee0 tx=0x7ff58000d3b0 comp rx=0 tx=0).stop 2024-04-25T01:45:35.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.848+0000 7ff58f027700 1 -- 172.21.15.78:0/1968849139 shutdown_connections 2024-04-25T01:45:35.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.848+0000 7ff58f027700 1 --2- 172.21.15.78:0/1968849139 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff5780a2c20 0x7ff5780a50e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:35.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.848+0000 7ff58f027700 1 --2- 172.21.15.78:0/1968849139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff588101130 0x7ff58819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:35.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.848+0000 7ff58f027700 1 --2- 172.21.15.78:0/1968849139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5881071a0 0x7ff588197770 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:35.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.848+0000 7ff58f027700 1 --2- 172.21.15.78:0/1968849139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff588101af0 0x7ff58819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:35.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.848+0000 7ff58f027700 1 -- 172.21.15.78:0/1968849139 >> 172.21.15.78:0/1968849139 conn(0x7ff5880780e0 msgr2=0x7ff5880ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:35.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.848+0000 7ff58f027700 1 -- 172.21.15.78:0/1968849139 shutdown_connections 2024-04-25T01:45:35.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:35.848+0000 7ff58f027700 1 -- 172.21.15.78:0/1968849139 wait complete. 2024-04-25T01:45:36.210 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:36 smithi078 ceph-mon[29469]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:45:36.210 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:36 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1968849139' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T01:45:36.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:36 smithi152 ceph-mon[32465]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:45:36.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:36 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1968849139' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T01:45:36.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:36 smithi196 ceph-mon[32978]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:45:36.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:36 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1968849139' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T01:45:36.607 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T01:45:36.608 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:45:36.608 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-04-25T01:45:36.608 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T01:45:36.608 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T01:45:36.608 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T01:45:36.608 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:45:36.608 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:45:36.608 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:45:36.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:45:36.608 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-04-25T01:45:36.625 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:45:36.625 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:45:36.625 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:45:36.625 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:45:36.625 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-04-25T01:45:36.625 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:45:36.625 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:45:36.625 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:45:36.625 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-04-25T01:45:36.625 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:45:36.625 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:45:36.626 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-04-25T01:45:36.626 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:45:36.626 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T01:45:36.626 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:45:36.626 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T01:45:36.626 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T01:45:36.626 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42044sec preferred_lft 42044sec 2024-04-25T01:45:36.626 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::7539:9f76:2904:c153/64 scope link noprefixroute 2024-04-25T01:45:36.626 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:45:36.626 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T01:45:36.627 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:45:36.627 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T01:45:36.627 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add name ceph-brx type bridge 2024-04-25T01:45:36.627 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr flush dev ceph-brx 2024-04-25T01:45:36.627 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set ceph-brx up 2024-04-25T01:45:36.627 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T01:45:36.627 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T01:45:36.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:45:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:45:36.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:45:36.764 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:45:36.787 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-04-25T01:45:36.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:45:36.788 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-04-25T01:45:36.844 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-04-25T01:45:36.844 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:45:36.844 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T01:45:36.844 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T01:45:36.844 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:45:36.845 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:45:36.845 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T01:45:36.845 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T01:45:36.845 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T01:45:36.845 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T01:45:36.845 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T01:45:36.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:45:36.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:45:36.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:45:36.969 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-04-25T01:45:36.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:45:36.984 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list-id 2024-04-25T01:45:37.039 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:45:37.039 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T01:45:37.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:45:37.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T01:45:37.039 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T01:45:37.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:45:37.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:45:37.156 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T01:45:37.156 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:45:37.156 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T01:45:37.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T01:45:37.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T01:45:37.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T01:45:37.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T01:45:37.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T01:45:37.156 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T01:45:37.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:45:37.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:45:37.266 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:45:37.266 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-04-25T01:45:37.267 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-04-25T01:45:37.267 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T01:45:37.267 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-04-25T01:45:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:45:37.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:45:37.343 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:45:37.343 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T01:45:37.357 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:45:37.358 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:45:37.358 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T01:45:37.412 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T01:45:37.412 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T01:45:37.412 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/99d6b083-a499-4291-8c2d-20d599de76b3 2024-04-25T01:45:37.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:45:37.413 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/99d6b083-a499-4291-8c2d-20d599de76b3 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-25T01:45:37.494 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-25T01:45:37.494 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-04-25T01:45:37.617 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-04-25T01:45:37.617 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-04-25T01:45:37.617 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.152:3300,172.21.15.196:3300" 2024-04-25T01:45:37.617 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@df36bea8-02a3-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/99d6b083-a499-4291-8c2d-20d599de76b3 2024-04-25T01:45:37.617 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.78:3300/172.21.15.152:3300/172.21.15.196:3300" will pass to kernel 2024-04-25T01:45:37.618 INFO:teuthology.orchestra.run.smithi078.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T01:45:37.618 INFO:teuthology.orchestra.run.smithi078.stdout: You just mounted a file system that supports labels which does not 2024-04-25T01:45:37.618 INFO:teuthology.orchestra.run.smithi078.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T01:45:37.618 INFO:teuthology.orchestra.run.smithi078.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T01:45:37.618 INFO:teuthology.orchestra.run.smithi078.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T01:45:37.620 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T01:45:37.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:45:37.620 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T01:45:37.686 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T01:45:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T01:45:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:45:37.783 INFO:teuthology.orchestra.run.smithi078.stdout:client25025 2024-04-25T01:45:37.783 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:45:37.783 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:45:37.798 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T01:45:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T01:45:37.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:45:37.893 INFO:teuthology.orchestra.run.smithi078.stdout:client25025 2024-04-25T01:45:37.894 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T01:45:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T01:45:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:45:37.949 INFO:teuthology.orchestra.run.smithi078.stdout:client25025 2024-04-25T01:45:37.949 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/df36bea8-02a3-11ef-bc93-c7b262605968.client25025/status') 2024-04-25T01:45:37.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:45:38.006 INFO:teuthology.orchestra.run.smithi078.stdout:instance: client.25025 (3)172.21.15.78:0/1143183544 2024-04-25T01:45:38.006 INFO:teuthology.orchestra.run.smithi078.stdout:blocklisted: false 2024-04-25T01:45:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records in 2024-04-25T01:45:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records out 2024-04-25T01:45:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:71 bytes copied, 2.187e-05 s, 3.2 MB/s 2024-04-25T01:45:38.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:45:38.013 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T01:45:38.024 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T01:45:38.024 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T01:45:38.024 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-25T01:45:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.132+0000 7f0209f77640 1 -- 172.21.15.78:0/2448055067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204104b90 msgr2=0x7f0204106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.132+0000 7f0209f77640 1 --2- 172.21.15.78:0/2448055067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204104b90 0x7f0204106f80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009900 tx=0x7f01f802a410 comp rx=0 tx=0).stop 2024-04-25T01:45:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.132+0000 7f0209f77640 1 -- 172.21.15.78:0/2448055067 shutdown_connections 2024-04-25T01:45:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.132+0000 7f0209f77640 1 --2- 172.21.15.78:0/2448055067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02041074c0 0x7f02041098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.132+0000 7f0209f77640 1 --2- 172.21.15.78:0/2448055067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204104b90 0x7f0204106f80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.132+0000 7f0209f77640 1 --2- 172.21.15.78:0/2448055067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0204102260 0x7f0204104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:38.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.132+0000 7f0209f77640 1 -- 172.21.15.78:0/2448055067 >> 172.21.15.78:0/2448055067 conn(0x7f02040fbc10 msgr2=0x7f02040fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.132+0000 7f0209f77640 1 -- 172.21.15.78:0/2448055067 shutdown_connections 2024-04-25T01:45:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.132+0000 7f0209f77640 1 -- 172.21.15.78:0/2448055067 wait complete. 2024-04-25T01:45:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f0209f77640 1 Processor -- start 2024-04-25T01:45:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f0209f77640 1 -- start start 2024-04-25T01:45:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f0209f77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204102260 0x7f0204199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f0209f77640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0204104b90 0x7f020419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f0209f77640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02041074c0 0x7f02041a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f0209f77640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f020410d470 con 0x7f0204102260 2024-04-25T01:45:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f0209f77640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f020410d2f0 con 0x7f02041074c0 2024-04-25T01:45:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f0209f77640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f020410d5f0 con 0x7f0204104b90 2024-04-25T01:45:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f02037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204102260 0x7f0204199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f02037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204102260 0x7f0204199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60470/0 (socket says 172.21.15.78:60470) 2024-04-25T01:45:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f02037fe640 1 -- 172.21.15.78:0/1349686945 learned_addr learned my addr 172.21.15.78:0/1349686945 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f0202ffd640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0204104b90 0x7f020419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f0203fff640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02041074c0 0x7f02041a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f02037fe640 1 -- 172.21.15.78:0/1349686945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0204104b90 msgr2=0x7f020419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f02037fe640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0204104b90 0x7f020419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f02037fe640 1 -- 172.21.15.78:0/1349686945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02041074c0 msgr2=0x7f02041a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f02037fe640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02041074c0 0x7f02041a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f02037fe640 1 -- 172.21.15.78:0/1349686945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02041a1880 con 0x7f0204102260 2024-04-25T01:45:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.133+0000 7f0203fff640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02041074c0 0x7f02041a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.134+0000 7f02037fe640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204102260 0x7f0204199c70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f01f400d950 tx=0x7f01f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.134+0000 7f0200ff9640 1 -- 172.21.15.78:0/1349686945 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4017070 con 0x7f0204102260 2024-04-25T01:45:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.134+0000 7f0200ff9640 1 -- 172.21.15.78:0/1349686945 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f4009c30 con 0x7f0204102260 2024-04-25T01:45:38.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.134+0000 7f0209f77640 1 -- 172.21.15.78:0/1349686945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02041a1b70 con 0x7f0204102260 2024-04-25T01:45:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.134+0000 7f0200ff9640 1 -- 172.21.15.78:0/1349686945 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f400f6b0 con 0x7f0204102260 2024-04-25T01:45:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.134+0000 7f0209f77640 1 -- 172.21.15.78:0/1349686945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02041a20c0 con 0x7f0204102260 2024-04-25T01:45:38.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.135+0000 7f0209f77640 1 -- 172.21.15.78:0/1349686945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0204108fb0 con 0x7f0204102260 2024-04-25T01:45:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.136+0000 7f0200ff9640 1 -- 172.21.15.78:0/1349686945 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f01f40109d0 con 0x7f0204102260 2024-04-25T01:45:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.137+0000 7f0200ff9640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f01ec0a2c30 0x7f01ec0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.137+0000 7f0202ffd640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f01ec0a2c30 0x7f01ec0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.137+0000 7f0200ff9640 1 -- 172.21.15.78:0/1349686945 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f01f40cf0c0 con 0x7f0204102260 2024-04-25T01:45:38.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.137+0000 7f0202ffd640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f01ec0a2c30 0x7f01ec0a50f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f01f802aa00 tx=0x7f01f80315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:38.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.140+0000 7f0200ff9640 1 -- 172.21.15.78:0/1349686945 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f01f4009de0 con 0x7f0204102260 2024-04-25T01:45:38.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:38 smithi078 ceph-mon[29469]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:45:38.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:38 smithi152 ceph-mon[32465]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:45:38.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:38.361+0000 7f0209f77640 1 -- 172.21.15.78:0/1349686945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f020402d170 con 0x7f0204102260 2024-04-25T01:45:38.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:38 smithi196 ceph-mon[32978]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:45:39.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.057+0000 7f0200ff9640 1 -- 172.21.15.78:0/1349686945 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7f01f409bc40 con 0x7f0204102260 2024-04-25T01:45:39.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.060+0000 7f0209f77640 1 -- 172.21.15.78:0/1349686945 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f01ec0a2c30 msgr2=0x7f01ec0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:39.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.060+0000 7f0209f77640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f01ec0a2c30 0x7f01ec0a50f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f01f802aa00 tx=0x7f01f80315b0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.060+0000 7f0209f77640 1 -- 172.21.15.78:0/1349686945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204102260 msgr2=0x7f0204199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:39.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.060+0000 7f0209f77640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204102260 0x7f0204199c70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f01f400d950 tx=0x7f01f400de10 comp rx=0 tx=0).stop 2024-04-25T01:45:39.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.060+0000 7f0209f77640 1 -- 172.21.15.78:0/1349686945 shutdown_connections 2024-04-25T01:45:39.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.060+0000 7f0209f77640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02041074c0 0x7f02041a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.060+0000 7f0209f77640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0204104b90 0x7f020419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.060+0000 7f0209f77640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f01ec0a2c30 0x7f01ec0a50f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.060+0000 7f0209f77640 1 --2- 172.21.15.78:0/1349686945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0204102260 0x7f0204199c70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.060+0000 7f0209f77640 1 -- 172.21.15.78:0/1349686945 >> 172.21.15.78:0/1349686945 conn(0x7f02040fbc10 msgr2=0x7f02041019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:39.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.061+0000 7f0209f77640 1 -- 172.21.15.78:0/1349686945 shutdown_connections 2024-04-25T01:45:39.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.061+0000 7f0209f77640 1 -- 172.21.15.78:0/1349686945 wait complete. 2024-04-25T01:45:39.070 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-25T01:45:39.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.186+0000 7f7cf4789640 1 -- 172.21.15.78:0/4130012792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec102300 msgr2=0x7f7cec108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.186+0000 7f7cf4789640 1 --2- 172.21.15.78:0/4130012792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec102300 0x7f7cec108b90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009ca0 tx=0x7f7ce402a8c0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.186+0000 7f7cf4789640 1 -- 172.21.15.78:0/4130012792 shutdown_connections 2024-04-25T01:45:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.186+0000 7f7cf4789640 1 --2- 172.21.15.78:0/4130012792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec102300 0x7f7cec108b90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.186+0000 7f7cf4789640 1 --2- 172.21.15.78:0/4130012792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cec101940 0x7f7cec101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.186+0000 7f7cf4789640 1 --2- 172.21.15.78:0/4130012792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cec100750 0x7f7cec100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.186+0000 7f7cf4789640 1 -- 172.21.15.78:0/4130012792 >> 172.21.15.78:0/4130012792 conn(0x7f7cec0fbcc0 msgr2=0x7f7cec0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.186+0000 7f7cf4789640 1 -- 172.21.15.78:0/4130012792 shutdown_connections 2024-04-25T01:45:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.186+0000 7f7cf4789640 1 -- 172.21.15.78:0/4130012792 wait complete. 2024-04-25T01:45:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf4789640 1 Processor -- start 2024-04-25T01:45:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf4789640 1 -- start start 2024-04-25T01:45:39.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf4789640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cec100750 0x7f7cec06aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:39.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf4789640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cec101940 0x7f7cec19ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:39.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf4789640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec102300 0x7f7cec1a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:39.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf4789640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec10c340 con 0x7f7cec102300 2024-04-25T01:45:39.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf4789640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec10c1c0 con 0x7f7cec100750 2024-04-25T01:45:39.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf4789640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec10c4c0 con 0x7f7cec101940 2024-04-25T01:45:39.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf2d26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec102300 0x7f7cec1a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:39.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf2d26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec102300 0x7f7cec1a0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60500/0 (socket says 172.21.15.78:60500) 2024-04-25T01:45:39.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf2d26640 1 -- 172.21.15.78:0/155602684 learned_addr learned my addr 172.21.15.78:0/155602684 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:45:39.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf2525640 1 --2- 172.21.15.78:0/155602684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cec100750 0x7f7cec06aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:39.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf1d24640 1 --2- 172.21.15.78:0/155602684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cec101940 0x7f7cec19ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:39.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf2d26640 1 -- 172.21.15.78:0/155602684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cec101940 msgr2=0x7f7cec19ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:39.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf2d26640 1 --2- 172.21.15.78:0/155602684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cec101940 0x7f7cec19ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf2d26640 1 -- 172.21.15.78:0/155602684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cec100750 msgr2=0x7f7cec06aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:39.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf2d26640 1 --2- 172.21.15.78:0/155602684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cec100750 0x7f7cec06aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf2d26640 1 -- 172.21.15.78:0/155602684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec1a1760 con 0x7f7cec102300 2024-04-25T01:45:39.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.187+0000 7f7cf2d26640 1 --2- 172.21.15.78:0/155602684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec102300 0x7f7cec1a0f70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4031ae0 tx=0x7f7ce4031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:39.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.188+0000 7f7ce37fe640 1 -- 172.21.15.78:0/155602684 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4003c30 con 0x7f7cec102300 2024-04-25T01:45:39.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.188+0000 7f7ce37fe640 1 -- 172.21.15.78:0/155602684 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce4003dd0 con 0x7f7cec102300 2024-04-25T01:45:39.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.188+0000 7f7ce37fe640 1 -- 172.21.15.78:0/155602684 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce402fcb0 con 0x7f7cec102300 2024-04-25T01:45:39.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.188+0000 7f7cf4789640 1 -- 172.21.15.78:0/155602684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec1a1990 con 0x7f7cec102300 2024-04-25T01:45:39.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.188+0000 7f7cf4789640 1 -- 172.21.15.78:0/155602684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cec1a1f30 con 0x7f7cec102300 2024-04-25T01:45:39.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.189+0000 7f7cf4789640 1 -- 172.21.15.78:0/155602684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cec1079a0 con 0x7f7cec102300 2024-04-25T01:45:39.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.190+0000 7f7ce37fe640 1 -- 172.21.15.78:0/155602684 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ce4004050 con 0x7f7cec102300 2024-04-25T01:45:39.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.190+0000 7f7ce37fe640 1 --2- 172.21.15.78:0/155602684 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7cd80a2ce0 0x7f7cd80a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:39.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.190+0000 7f7ce37fe640 1 -- 172.21.15.78:0/155602684 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f7ce40f0860 con 0x7f7cec102300 2024-04-25T01:45:39.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.190+0000 7f7cf2525640 1 --2- 172.21.15.78:0/155602684 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7cd80a2ce0 0x7f7cd80a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:39.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.191+0000 7f7cf2525640 1 --2- 172.21.15.78:0/155602684 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7cd80a2ce0 0x7f7cd80a51a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7cec1017a0 tx=0x7f7ce8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:39.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.194+0000 7f7ce37fe640 1 -- 172.21.15.78:0/155602684 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7ce40bd3e0 con 0x7f7cec102300 2024-04-25T01:45:39.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:39 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1349686945' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T01:45:39.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:39 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1349686945' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T01:45:39.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.394+0000 7f7cf4789640 1 -- 172.21.15.78:0/155602684 --> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f7cec061b80 con 0x7f7cd80a2ce0 2024-04-25T01:45:39.394 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:39 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1349686945' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T01:45:39.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.417+0000 7f7ce37fe640 1 -- 172.21.15.78:0/155602684 <== mgr.14148 v2:172.21.15.78:6800/2190983290 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f7cec061b80 con 0x7f7cd80a2ce0 2024-04-25T01:45:39.411 INFO:teuthology.orchestra.run.smithi078.stdout:{} 2024-04-25T01:45:39.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.419+0000 7f7cf4789640 1 -- 172.21.15.78:0/155602684 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7cd80a2ce0 msgr2=0x7f7cd80a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:39.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.419+0000 7f7cf4789640 1 --2- 172.21.15.78:0/155602684 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7cd80a2ce0 0x7f7cd80a51a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7cec1017a0 tx=0x7f7ce8005f90 comp rx=0 tx=0).stop 2024-04-25T01:45:39.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.419+0000 7f7cf4789640 1 -- 172.21.15.78:0/155602684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec102300 msgr2=0x7f7cec1a0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:39.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.419+0000 7f7cf4789640 1 --2- 172.21.15.78:0/155602684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec102300 0x7f7cec1a0f70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4031ae0 tx=0x7f7ce4031ee0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.419+0000 7f7cf4789640 1 -- 172.21.15.78:0/155602684 shutdown_connections 2024-04-25T01:45:39.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.419+0000 7f7cf4789640 1 --2- 172.21.15.78:0/155602684 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7cd80a2ce0 0x7f7cd80a51a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.419+0000 7f7cf4789640 1 --2- 172.21.15.78:0/155602684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec102300 0x7f7cec1a0f70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.419+0000 7f7cf4789640 1 --2- 172.21.15.78:0/155602684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cec101940 0x7f7cec19ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.419+0000 7f7cf4789640 1 --2- 172.21.15.78:0/155602684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7cec100750 0x7f7cec06aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:39.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.419+0000 7f7cf4789640 1 -- 172.21.15.78:0/155602684 >> 172.21.15.78:0/155602684 conn(0x7f7cec0fbcc0 msgr2=0x7f7cec104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:39.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.419+0000 7f7cf4789640 1 -- 172.21.15.78:0/155602684 shutdown_connections 2024-04-25T01:45:39.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:45:39.419+0000 7f7cf4789640 1 -- 172.21.15.78:0/155602684 wait complete. 2024-04-25T01:45:39.423 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T01:45:39.434 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T01:45:39.446 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:45:39.447 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T01:45:39.447 INFO:tasks.workunit:timeout=3h 2024-04-25T01:45:39.447 INFO:tasks.workunit:cleanup=True 2024-04-25T01:45:39.447 DEBUG:teuthology.orchestra.run.smithi078:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T01:45:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T01:45:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T01:45:39.462 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-25T01:45:39.462 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:45:39.463 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T01:45:39.463 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-25 01:43:44.479429974 +0000 2024-04-25T01:45:39.463 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-25 01:43:44.479429974 +0000 2024-04-25T01:45:39.463 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-25 01:45:37.690956373 +0000 2024-04-25T01:45:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-04-25 01:43:44.479429974 +0000 2024-04-25T01:45:39.463 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T01:45:39.463 DEBUG:teuthology.orchestra.run.smithi078:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T01:45:39.536 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:45:39.590 INFO:tasks.workunit.client.0.smithi078.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T01:45:39.758 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:45:39 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:45:39.399+0000 7f41f3cce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:45:39.758 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:45:39 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:45:39.399+0000 7f41f3cce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:45:39.759 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:45:39 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:45:39.399+0000 7f41f3cce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:45:39.759 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:45:39 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:45:39.399+0000 7f41f3cce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:45:39.759 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:45:39 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:45:39.399+0000 7f41f3cce700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T01:45:40.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:40 smithi196 ceph-mon[32978]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:45:40.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:40 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1349686945' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T01:45:40.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:40 smithi196 ceph-mon[32978]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T01:45:40.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:40 smithi078 ceph-mon[29469]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:45:40.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:40 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1349686945' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T01:45:40.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:40 smithi078 ceph-mon[29469]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T01:45:40.550 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:40 smithi152 ceph-mon[32465]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:45:40.550 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:40 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1349686945' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T01:45:40.550 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:40 smithi152 ceph-mon[32465]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T01:45:41.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:41 smithi196 ceph-mon[32978]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:41.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:41 smithi078 ceph-mon[29469]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:41.550 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:41 smithi152 ceph-mon[32465]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:42.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:42 smithi196 ceph-mon[32978]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:45:42.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:42 smithi196 ceph-mon[32978]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T01:45:42.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:42 smithi078 ceph-mon[29469]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:45:42.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:42 smithi078 ceph-mon[29469]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T01:45:42.550 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:42 smithi152 ceph-mon[32465]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:45:42.550 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:42 smithi152 ceph-mon[32465]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T01:45:43.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:43 smithi196 ceph-mon[32978]: pgmap v367: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T01:45:43.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:43 smithi078 ceph-mon[29469]: pgmap v367: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T01:45:43.550 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:43 smithi152 ceph-mon[32465]: pgmap v367: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T01:45:46.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:45 smithi078 ceph-mon[29469]: pgmap v368: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-25T01:45:46.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:45 smithi152 ceph-mon[32465]: pgmap v368: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-25T01:45:46.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:45 smithi196 ceph-mon[32978]: pgmap v368: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-25T01:45:48.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:47 smithi078 ceph-mon[29469]: pgmap v369: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-25T01:45:48.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:47 smithi152 ceph-mon[32465]: pgmap v369: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-25T01:45:48.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:47 smithi196 ceph-mon[32978]: pgmap v369: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-25T01:45:50.244 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:49 smithi078 ceph-mon[29469]: pgmap v370: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-25T01:45:50.245 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:49 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:50.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:49 smithi152 ceph-mon[32465]: pgmap v370: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-25T01:45:50.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:49 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:50.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:49 smithi196 ceph-mon[32978]: pgmap v370: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-25T01:45:50.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:49 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:52.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:51 smithi078 ceph-mon[29469]: pgmap v371: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-25T01:45:52.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:52.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:45:52.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:51 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:51 smithi152 ceph-mon[32465]: pgmap v371: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-25T01:45:52.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:52.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:45:52.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:51 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:51 smithi196 ceph-mon[32978]: pgmap v371: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-25T01:45:52.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:52.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:52.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:45:52.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:51 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:45:54.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:53 smithi078 ceph-mon[29469]: pgmap v372: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-25T01:45:54.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:53 smithi152 ceph-mon[32465]: pgmap v372: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-25T01:45:54.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:53 smithi196 ceph-mon[32978]: pgmap v372: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-25T01:45:56.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:56 smithi078 ceph-mon[29469]: pgmap v373: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 11 op/s 2024-04-25T01:45:56.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:56 smithi152 ceph-mon[32465]: pgmap v373: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 11 op/s 2024-04-25T01:45:56.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:56 smithi196 ceph-mon[32978]: pgmap v373: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 11 op/s 2024-04-25T01:45:58.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:45:58 smithi078 ceph-mon[29469]: pgmap v374: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T01:45:58.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:45:58 smithi152 ceph-mon[32465]: pgmap v374: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T01:45:58.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:45:58 smithi196 ceph-mon[32978]: pgmap v374: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T01:46:00.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:00 smithi078 ceph-mon[29469]: pgmap v375: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T01:46:00.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:00 smithi152 ceph-mon[32465]: pgmap v375: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T01:46:00.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:00 smithi196 ceph-mon[32978]: pgmap v375: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T01:46:02.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:02 smithi078 ceph-mon[29469]: pgmap v376: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-25T01:46:02.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:02 smithi152 ceph-mon[32465]: pgmap v376: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-25T01:46:02.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:02 smithi196 ceph-mon[32978]: pgmap v376: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-25T01:46:04.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:04 smithi078 ceph-mon[29469]: pgmap v377: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-25T01:46:04.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:04 smithi152 ceph-mon[32465]: pgmap v377: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-25T01:46:04.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:04 smithi196 ceph-mon[32978]: pgmap v377: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-25T01:46:06.258 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:06 smithi078 ceph-mon[29469]: pgmap v378: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:46:06.300 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:06 smithi152 ceph-mon[32465]: pgmap v378: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:46:06.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:06 smithi196 ceph-mon[32978]: pgmap v378: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:46:08.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:08 smithi152 ceph-mon[32465]: pgmap v379: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:46:08.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:08 smithi196 ceph-mon[32978]: pgmap v379: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:46:08.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:08 smithi078 ceph-mon[29469]: pgmap v379: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:46:10.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:10 smithi152 ceph-mon[32465]: pgmap v380: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:46:10.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:10 smithi196 ceph-mon[32978]: pgmap v380: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:46:10.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:10 smithi078 ceph-mon[29469]: pgmap v380: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:46:12.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:12 smithi152 ceph-mon[32465]: pgmap v381: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:46:12.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:12 smithi196 ceph-mon[32978]: pgmap v381: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:46:12.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:12 smithi078 ceph-mon[29469]: pgmap v381: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:46:14.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:14 smithi152 ceph-mon[32465]: pgmap v382: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:46:14.368 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:14 smithi196 ceph-mon[32978]: pgmap v382: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:46:14.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:14 smithi078 ceph-mon[29469]: pgmap v382: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:46:16.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:16 smithi152 ceph-mon[32465]: pgmap v383: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:46:16.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:16 smithi196 ceph-mon[32978]: pgmap v383: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:46:16.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:16 smithi078 ceph-mon[29469]: pgmap v383: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:46:18.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:18 smithi152 ceph-mon[32465]: pgmap v384: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T01:46:18.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:18 smithi196 ceph-mon[32978]: pgmap v384: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T01:46:18.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:18 smithi078 ceph-mon[29469]: pgmap v384: 129 pgs: 129 active+clean; 663 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T01:46:20.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:20 smithi152 ceph-mon[32465]: pgmap v385: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T01:46:20.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:20 smithi196 ceph-mon[32978]: pgmap v385: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T01:46:20.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:20 smithi078 ceph-mon[29469]: pgmap v385: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T01:46:22.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:22 smithi152 ceph-mon[32465]: pgmap v386: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T01:46:22.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:22 smithi196 ceph-mon[32978]: pgmap v386: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T01:46:22.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:22 smithi078 ceph-mon[29469]: pgmap v386: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T01:46:24.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:24 smithi152 ceph-mon[32465]: pgmap v387: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T01:46:24.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:24 smithi196 ceph-mon[32978]: pgmap v387: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T01:46:24.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:24 smithi078 ceph-mon[29469]: pgmap v387: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T01:46:26.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:26 smithi152 ceph-mon[32465]: pgmap v388: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T01:46:26.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:26 smithi196 ceph-mon[32978]: pgmap v388: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T01:46:26.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:26 smithi078 ceph-mon[29469]: pgmap v388: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T01:46:28.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:28 smithi152 ceph-mon[32465]: pgmap v389: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T01:46:28.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:28 smithi196 ceph-mon[32978]: pgmap v389: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T01:46:28.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:28 smithi078 ceph-mon[29469]: pgmap v389: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T01:46:30.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:30 smithi152 ceph-mon[32465]: pgmap v390: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T01:46:30.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:30 smithi196 ceph-mon[32978]: pgmap v390: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T01:46:30.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:30 smithi078 ceph-mon[29469]: pgmap v390: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T01:46:32.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:32 smithi152 ceph-mon[32465]: pgmap v391: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:46:32.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:32 smithi196 ceph-mon[32978]: pgmap v391: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:46:32.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:32 smithi078 ceph-mon[29469]: pgmap v391: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:46:34.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:34 smithi152 ceph-mon[32465]: pgmap v392: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:46:34.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:34 smithi196 ceph-mon[32978]: pgmap v392: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:46:34.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:34 smithi078 ceph-mon[29469]: pgmap v392: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:46:36.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:36 smithi152 ceph-mon[32465]: pgmap v393: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:46:36.310 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:36 smithi078 ceph-mon[29469]: pgmap v393: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:46:36.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:36 smithi196 ceph-mon[32978]: pgmap v393: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:46:38.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:38 smithi152 ceph-mon[32465]: pgmap v394: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:46:38.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:38 smithi196 ceph-mon[32978]: pgmap v394: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:46:38.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:38 smithi078 ceph-mon[29469]: pgmap v394: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:46:40.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:40 smithi196 ceph-mon[32978]: pgmap v395: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:46:40.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:40 smithi078 ceph-mon[29469]: pgmap v395: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:46:40.551 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:40 smithi152 ceph-mon[32465]: pgmap v395: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T01:46:42.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:42 smithi196 ceph-mon[32978]: pgmap v396: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:46:42.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:42 smithi078 ceph-mon[29469]: pgmap v396: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:46:42.551 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:42 smithi152 ceph-mon[32465]: pgmap v396: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T01:46:44.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:44 smithi196 ceph-mon[32978]: pgmap v397: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:46:44.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:44 smithi078 ceph-mon[29469]: pgmap v397: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:46:44.551 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:44 smithi152 ceph-mon[32465]: pgmap v397: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T01:46:46.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:46 smithi196 ceph-mon[32978]: pgmap v398: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:46:46.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:46 smithi078 ceph-mon[29469]: pgmap v398: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:46:46.551 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:46 smithi152 ceph-mon[32465]: pgmap v398: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:46:47.218 INFO:tasks.workunit.client.0.smithi078.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-25T01:46:47.218 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-04-25T01:46:47.218 INFO:tasks.workunit.client.0.smithi078.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T01:46:47.218 INFO:tasks.workunit.client.0.smithi078.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T01:46:47.218 INFO:tasks.workunit.client.0.smithi078.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T01:46:47.218 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-04-25T01:46:47.218 INFO:tasks.workunit.client.0.smithi078.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T01:46:47.219 INFO:tasks.workunit.client.0.smithi078.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T01:46:47.219 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-04-25T01:46:47.219 INFO:tasks.workunit.client.0.smithi078.stderr: git switch -c 2024-04-25T01:46:47.219 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-04-25T01:46:47.219 INFO:tasks.workunit.client.0.smithi078.stderr:Or undo this operation with: 2024-04-25T01:46:47.219 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-04-25T01:46:47.219 INFO:tasks.workunit.client.0.smithi078.stderr: git switch - 2024-04-25T01:46:47.219 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-04-25T01:46:47.219 INFO:tasks.workunit.client.0.smithi078.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T01:46:47.219 INFO:tasks.workunit.client.0.smithi078.stderr: 2024-04-25T01:46:47.219 INFO:tasks.workunit.client.0.smithi078.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-25T01:46:47.222 DEBUG:teuthology.orchestra.run.smithi078:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T01:46:47.290 INFO:tasks.workunit.client.0.smithi078.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T01:46:47.292 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T01:46:47.293 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T01:46:47.830 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T01:46:47.885 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T01:46:47.925 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T01:46:47.926 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T01:46:47.926 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T01:46:47.964 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T01:46:47.968 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:46:47.968 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T01:46:48.023 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-04-25T01:46:48.024 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-04-25T01:46:48.024 DEBUG:teuthology.orchestra.run.smithi078:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-04-25T01:46:48.085 INFO:tasks.workunit.client.0.smithi078.stdout:test large (16MB) dio write 2024-04-25T01:46:48.086 INFO:tasks.workunit.client.0.smithi078.stderr:+ echo 'test large (16MB) dio write' 2024-04-25T01:46:48.086 INFO:tasks.workunit.client.0.smithi078.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-04-25T01:46:48.170 INFO:tasks.workunit.client.0.smithi078.stderr:1+0 records in 2024-04-25T01:46:48.170 INFO:tasks.workunit.client.0.smithi078.stderr:1+0 records out 2024-04-25T01:46:48.170 INFO:tasks.workunit.client.0.smithi078.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.0819228 s, 205 MB/s 2024-04-25T01:46:48.171 INFO:tasks.workunit.client.0.smithi078.stderr:+ echo OK 2024-04-25T01:46:48.172 INFO:tasks.workunit.client.0.smithi078.stdout:OK 2024-04-25T01:46:48.173 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T01:46:48.173 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T01:46:48.250 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-04-25T01:46:48.250 DEBUG:teuthology.orchestra.run.smithi078:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-04-25T01:46:48.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:48 smithi196 ceph-mon[32978]: pgmap v399: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:46:48.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:48 smithi196 ceph-mon[32978]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-25T01:46:48.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:48 smithi078 ceph-mon[29469]: pgmap v399: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:46:48.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:48 smithi078 ceph-mon[29469]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-25T01:46:48.551 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:48 smithi152 ceph-mon[32465]: pgmap v399: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T01:46:48.551 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:48 smithi152 ceph-mon[32465]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-25T01:46:49.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:49 smithi196 ceph-mon[32978]: pgmap v400: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:46:49.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:49 smithi078 ceph-mon[29469]: pgmap v400: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:46:49.551 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:49 smithi152 ceph-mon[32465]: pgmap v400: 129 pgs: 129 active+clean; 668 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T01:46:52.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:52 smithi078 ceph-mon[29469]: pgmap v401: 129 pgs: 129 active+clean; 1.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-04-25T01:46:52.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:52 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:46:52.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:52 smithi152 ceph-mon[32465]: pgmap v401: 129 pgs: 129 active+clean; 1.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-04-25T01:46:52.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:52 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:46:52.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:52 smithi196 ceph-mon[32978]: pgmap v401: 129 pgs: 129 active+clean; 1.0 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-04-25T01:46:52.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:52 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:46:53.760 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.760 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.760 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.760 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.760 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.760 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.760 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:53 smithi078 ceph-mon[29469]: pgmap v402: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-04-25T01:46:53.760 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:46:53.760 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:46:53.761 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:53 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.801 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.802 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.802 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.802 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.802 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.802 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.802 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:53 smithi152 ceph-mon[32465]: pgmap v402: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-04-25T01:46:53.802 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:46:53.802 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:46:53.802 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:53 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.869 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.869 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.869 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.869 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.870 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.870 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:53.870 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:53 smithi196 ceph-mon[32978]: pgmap v402: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-04-25T01:46:53.870 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:46:53.870 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:46:53.870 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:53 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:46:56.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:56 smithi078 ceph-mon[29469]: pgmap v403: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 730 op/s 2024-04-25T01:46:56.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:56 smithi152 ceph-mon[32465]: pgmap v403: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 730 op/s 2024-04-25T01:46:56.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:56 smithi196 ceph-mon[32978]: pgmap v403: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 730 op/s 2024-04-25T01:46:57.995 INFO:tasks.workunit.client.0.smithi078.stdout:.......... 2024-04-25T01:46:57.995 INFO:tasks.workunit.client.0.smithi078.stdout:do_o_direct_reads: SUCCESS 2024-04-25T01:46:57.999 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T01:46:58.000 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T01:46:58.033 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-04-25T01:46:58.033 DEBUG:teuthology.orchestra.run.smithi078:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-04-25T01:46:58.092 INFO:tasks.workunit.client.0.smithi078.stdout:test read from hole 2024-04-25T01:46:58.093 INFO:tasks.workunit.client.0.smithi078.stderr:+ echo 'test read from hole' 2024-04-25T01:46:58.093 INFO:tasks.workunit.client.0.smithi078.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-04-25T01:46:58.094 INFO:tasks.workunit.client.0.smithi078.stderr:0+0 records in 2024-04-25T01:46:58.095 INFO:tasks.workunit.client.0.smithi078.stderr:0+0 records out 2024-04-25T01:46:58.095 INFO:tasks.workunit.client.0.smithi078.stderr:0 bytes copied, 0.000100429 s, 0.0 kB/s 2024-04-25T01:46:58.095 INFO:tasks.workunit.client.0.smithi078.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-04-25T01:46:58.097 INFO:tasks.workunit.client.0.smithi078.stderr:2+0 records in 2024-04-25T01:46:58.097 INFO:tasks.workunit.client.0.smithi078.stderr:2+0 records out 2024-04-25T01:46:58.097 INFO:tasks.workunit.client.0.smithi078.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00135807 s, 754 kB/s 2024-04-25T01:46:58.097 INFO:tasks.workunit.client.0.smithi078.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-04-25T01:46:58.097 INFO:tasks.workunit.client.0.smithi078.stderr:2+0 records in 2024-04-25T01:46:58.097 INFO:tasks.workunit.client.0.smithi078.stderr:2+0 records out 2024-04-25T01:46:58.098 INFO:tasks.workunit.client.0.smithi078.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 4.0106e-05 s, 25.5 MB/s 2024-04-25T01:46:58.098 INFO:tasks.workunit.client.0.smithi078.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-04-25T01:46:58.098 INFO:tasks.workunit.client.0.smithi078.stdout:other thing 2024-04-25T01:46:58.099 INFO:tasks.workunit.client.0.smithi078.stderr:+ echo 'other thing' 2024-04-25T01:46:58.099 INFO:tasks.workunit.client.0.smithi078.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-04-25T01:46:58.099 INFO:tasks.workunit.client.0.smithi078.stderr:1+0 records in 2024-04-25T01:46:58.099 INFO:tasks.workunit.client.0.smithi078.stderr:1+0 records out 2024-04-25T01:46:58.099 INFO:tasks.workunit.client.0.smithi078.stderr:500 bytes copied, 5.2983e-05 s, 9.4 MB/s 2024-04-25T01:46:58.099 INFO:tasks.workunit.client.0.smithi078.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-04-25T01:46:58.107 INFO:tasks.workunit.client.0.smithi078.stderr:0+1 records in 2024-04-25T01:46:58.107 INFO:tasks.workunit.client.0.smithi078.stderr:0+1 records out 2024-04-25T01:46:58.107 INFO:tasks.workunit.client.0.smithi078.stderr:500 bytes copied, 0.00625819 s, 79.9 kB/s 2024-04-25T01:46:58.107 INFO:tasks.workunit.client.0.smithi078.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-04-25T01:46:58.113 INFO:tasks.workunit.client.0.smithi078.stderr:0+1 records in 2024-04-25T01:46:58.113 INFO:tasks.workunit.client.0.smithi078.stderr:0+1 records out 2024-04-25T01:46:58.113 INFO:tasks.workunit.client.0.smithi078.stderr:500 bytes copied, 0.00573958 s, 87.1 kB/s 2024-04-25T01:46:58.113 INFO:tasks.workunit.client.0.smithi078.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-04-25T01:46:58.114 INFO:tasks.workunit.client.0.smithi078.stdout:OK 2024-04-25T01:46:58.115 INFO:tasks.workunit.client.0.smithi078.stderr:+ echo OK 2024-04-25T01:46:58.115 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T01:46:58.115 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T01:46:58.195 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-04-25T01:46:58.196 DEBUG:teuthology.orchestra.run.smithi078:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-04-25T01:46:58.259 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:46:58 smithi078 ceph-mon[29469]: pgmap v404: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 729 op/s 2024-04-25T01:46:58.270 INFO:tasks.workunit.client.0.smithi078.stdout:writing first 3 bytes of 10k file 2024-04-25T01:46:58.270 INFO:tasks.workunit.client.0.smithi078.stdout:reading O_DIRECT 2024-04-25T01:46:58.270 INFO:tasks.workunit.client.0.smithi078.stdout:got 10000 2024-04-25T01:46:58.270 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T01:46:58.270 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T01:46:58.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:46:58 smithi152 ceph-mon[32465]: pgmap v404: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 729 op/s 2024-04-25T01:46:58.347 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-04-25T01:46:58.347 DEBUG:teuthology.orchestra.run.smithi078:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-04-25T01:46:58.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:46:58 smithi196 ceph-mon[32978]: pgmap v404: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 729 op/s 2024-04-25T01:46:59.651 INFO:tasks.workunit.client.0.smithi078.stdout:writing pattern 2024-04-25T01:46:59.651 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-04-25T01:46:59.651 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-04-25T01:46:59.651 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-04-25T01:46:59.651 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-04-25T01:46:59.651 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-04-25T01:46:59.651 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-04-25T01:46:59.651 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-04-25T01:46:59.651 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-04-25T01:46:59.651 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-04-25T01:46:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-04-25T01:46:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-04-25T01:46:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-04-25T01:46:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-04-25T01:46:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-04-25T01:46:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-04-25T01:46:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-04-25T01:46:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-04-25T01:46:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-04-25T01:46:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-04-25T01:46:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-04-25T01:46:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-04-25T01:46:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-04-25T01:46:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-04-25T01:46:59.653 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-04-25T01:46:59.653 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-04-25T01:46:59.653 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-04-25T01:46:59.653 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-04-25T01:46:59.653 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-04-25T01:46:59.653 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-04-25T01:46:59.653 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-04-25T01:46:59.653 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-04-25T01:46:59.653 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-04-25T01:46:59.653 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-04-25T01:46:59.653 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-04-25T01:46:59.653 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-04-25T01:46:59.653 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-04-25T01:46:59.653 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-04-25T01:46:59.654 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-04-25T01:46:59.654 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-04-25T01:46:59.654 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-04-25T01:46:59.654 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-04-25T01:46:59.654 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-04-25T01:46:59.654 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-04-25T01:46:59.654 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-04-25T01:46:59.654 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-04-25T01:46:59.654 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-04-25T01:46:59.654 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-04-25T01:46:59.654 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-04-25T01:46:59.654 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-04-25T01:46:59.654 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-04-25T01:46:59.654 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-04-25T01:46:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-04-25T01:46:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-04-25T01:46:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-04-25T01:46:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-04-25T01:46:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-04-25T01:46:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-04-25T01:46:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-04-25T01:46:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-04-25T01:46:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-04-25T01:46:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-04-25T01:46:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-04-25T01:46:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-04-25T01:46:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-04-25T01:46:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-04-25T01:46:59.656 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-04-25T01:46:59.656 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-04-25T01:46:59.656 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-04-25T01:46:59.656 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-04-25T01:46:59.656 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-04-25T01:46:59.656 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-04-25T01:46:59.656 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-04-25T01:46:59.656 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-04-25T01:46:59.656 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-04-25T01:46:59.656 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-04-25T01:46:59.656 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-04-25T01:46:59.656 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-04-25T01:46:59.656 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-04-25T01:46:59.656 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-04-25T01:46:59.657 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-04-25T01:46:59.657 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-04-25T01:46:59.657 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-04-25T01:46:59.657 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-04-25T01:46:59.657 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-04-25T01:46:59.657 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-04-25T01:46:59.657 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-04-25T01:46:59.657 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-04-25T01:46:59.697 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-04-25T01:46:59.697 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-04-25T01:46:59.697 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-04-25T01:46:59.698 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-04-25T01:46:59.698 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-04-25T01:46:59.698 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-04-25T01:46:59.698 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-04-25T01:46:59.698 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-04-25T01:46:59.698 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-04-25T01:46:59.698 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-04-25T01:46:59.698 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-04-25T01:46:59.698 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-04-25T01:46:59.698 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-04-25T01:46:59.698 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-04-25T01:46:59.698 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-04-25T01:46:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-04-25T01:46:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-04-25T01:46:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-04-25T01:46:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-04-25T01:46:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-04-25T01:46:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-04-25T01:46:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-04-25T01:46:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-04-25T01:46:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-04-25T01:46:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-04-25T01:46:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-04-25T01:46:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-04-25T01:46:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-04-25T01:46:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-04-25T01:46:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-04-25T01:46:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-04-25T01:46:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-04-25T01:46:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-04-25T01:46:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-04-25T01:46:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-04-25T01:46:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-04-25T01:46:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-04-25T01:46:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-04-25T01:46:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-04-25T01:46:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-04-25T01:46:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-04-25T01:46:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-04-25T01:46:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-04-25T01:46:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-04-25T01:46:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-04-25T01:46:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-04-25T01:46:59.720 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-04-25T01:46:59.720 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-04-25T01:46:59.720 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-04-25T01:46:59.721 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-04-25T01:46:59.721 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-04-25T01:46:59.721 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-04-25T01:46:59.721 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-04-25T01:46:59.721 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-04-25T01:46:59.721 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-04-25T01:46:59.721 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-04-25T01:46:59.721 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-04-25T01:46:59.721 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-04-25T01:46:59.721 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-04-25T01:46:59.721 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-04-25T01:46:59.721 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-04-25T01:46:59.721 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-04-25T01:46:59.722 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-04-25T01:46:59.722 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-04-25T01:46:59.722 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-04-25T01:46:59.722 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-04-25T01:46:59.722 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-04-25T01:46:59.722 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-04-25T01:46:59.722 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-04-25T01:46:59.722 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-04-25T01:46:59.722 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-04-25T01:46:59.722 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-04-25T01:46:59.722 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-04-25T01:46:59.722 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-04-25T01:46:59.722 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-04-25T01:46:59.722 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-04-25T01:46:59.723 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-04-25T01:46:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-04-25T01:46:59.725 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-04-25T01:46:59.726 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-04-25T01:46:59.726 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-04-25T01:46:59.726 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-04-25T01:46:59.726 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-04-25T01:46:59.726 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-04-25T01:46:59.726 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-04-25T01:46:59.726 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-04-25T01:46:59.726 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-04-25T01:46:59.726 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-04-25T01:46:59.726 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-04-25T01:46:59.726 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-04-25T01:46:59.726 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-04-25T01:46:59.747 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-04-25T01:46:59.747 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-04-25T01:46:59.747 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-04-25T01:46:59.747 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-04-25T01:46:59.747 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-04-25T01:46:59.747 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-04-25T01:46:59.747 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-04-25T01:46:59.747 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-04-25T01:46:59.747 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-04-25T01:46:59.747 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-04-25T01:46:59.747 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-04-25T01:46:59.747 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-04-25T01:46:59.748 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-04-25T01:46:59.749 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-04-25T01:46:59.749 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-04-25T01:46:59.749 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-04-25T01:46:59.749 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-04-25T01:46:59.749 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-04-25T01:46:59.749 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-04-25T01:46:59.749 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-04-25T01:46:59.749 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-04-25T01:46:59.749 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-04-25T01:46:59.749 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-04-25T01:46:59.749 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-04-25T01:46:59.749 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-04-25T01:46:59.749 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-04-25T01:46:59.749 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-04-25T01:46:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-04-25T01:46:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-04-25T01:46:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-04-25T01:46:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-04-25T01:46:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-04-25T01:46:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-04-25T01:46:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-04-25T01:46:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-04-25T01:46:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-04-25T01:46:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-04-25T01:46:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-04-25T01:46:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-04-25T01:46:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-04-25T01:46:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-04-25T01:46:59.751 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-04-25T01:46:59.752 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-04-25T01:46:59.753 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-04-25T01:46:59.753 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-04-25T01:46:59.785 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-04-25T01:46:59.785 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-04-25T01:46:59.785 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-04-25T01:46:59.786 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-04-25T01:46:59.786 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-04-25T01:46:59.786 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-04-25T01:46:59.786 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-04-25T01:46:59.786 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-04-25T01:46:59.786 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-04-25T01:46:59.786 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-04-25T01:46:59.786 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-04-25T01:46:59.786 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-04-25T01:46:59.786 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-04-25T01:46:59.786 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-04-25T01:46:59.786 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-04-25T01:46:59.786 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-04-25T01:46:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-04-25T01:46:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-04-25T01:46:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-04-25T01:46:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-04-25T01:46:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-04-25T01:46:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-04-25T01:46:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-04-25T01:46:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-04-25T01:46:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-04-25T01:46:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-04-25T01:46:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-04-25T01:46:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-04-25T01:46:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-04-25T01:46:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-04-25T01:46:59.788 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-04-25T01:46:59.788 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-04-25T01:46:59.788 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-04-25T01:46:59.788 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-04-25T01:46:59.788 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-04-25T01:46:59.788 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-04-25T01:46:59.788 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-04-25T01:46:59.788 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-04-25T01:46:59.788 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-04-25T01:46:59.788 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-04-25T01:46:59.788 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-04-25T01:46:59.788 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-04-25T01:46:59.788 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-04-25T01:46:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-04-25T01:46:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-04-25T01:46:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-04-25T01:46:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-04-25T01:46:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-04-25T01:46:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-04-25T01:46:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-04-25T01:46:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-04-25T01:46:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-04-25T01:46:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-04-25T01:46:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-04-25T01:46:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-04-25T01:46:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-04-25T01:46:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-04-25T01:46:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-04-25T01:46:59.791 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-04-25T01:46:59.791 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-04-25T01:46:59.791 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-04-25T01:46:59.791 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-04-25T01:46:59.791 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-04-25T01:46:59.791 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-04-25T01:46:59.791 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-04-25T01:46:59.791 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-04-25T01:46:59.791 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-04-25T01:46:59.791 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-04-25T01:46:59.791 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-04-25T01:46:59.791 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-04-25T01:46:59.818 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-04-25T01:46:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-04-25T01:46:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-04-25T01:46:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-04-25T01:46:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-04-25T01:46:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-04-25T01:46:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-04-25T01:46:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-04-25T01:46:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-04-25T01:46:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-04-25T01:46:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-04-25T01:46:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-04-25T01:46:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-04-25T01:46:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-04-25T01:46:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-04-25T01:46:59.820 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-04-25T01:46:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-04-25T01:46:59.822 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-04-25T01:46:59.822 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-04-25T01:46:59.822 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-04-25T01:46:59.822 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-04-25T01:46:59.822 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-04-25T01:46:59.822 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-04-25T01:46:59.822 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-04-25T01:46:59.822 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-04-25T01:46:59.822 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-04-25T01:46:59.822 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-04-25T01:46:59.822 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-04-25T01:46:59.822 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-04-25T01:46:59.822 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-04-25T01:46:59.822 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-04-25T01:46:59.823 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-04-25T01:46:59.823 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-04-25T01:46:59.823 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-04-25T01:46:59.823 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-04-25T01:46:59.823 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-04-25T01:46:59.823 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-04-25T01:46:59.823 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-04-25T01:46:59.823 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-04-25T01:46:59.823 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-04-25T01:46:59.823 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-04-25T01:46:59.823 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-04-25T01:46:59.823 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-04-25T01:46:59.823 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-04-25T01:46:59.823 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-04-25T01:46:59.824 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-04-25T01:46:59.824 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-04-25T01:46:59.824 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-04-25T01:46:59.824 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-04-25T01:46:59.824 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-04-25T01:46:59.824 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-04-25T01:46:59.824 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-04-25T01:46:59.824 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-04-25T01:46:59.824 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-04-25T01:46:59.824 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-04-25T01:46:59.824 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-04-25T01:46:59.842 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-04-25T01:46:59.842 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-04-25T01:46:59.842 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-04-25T01:46:59.842 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-04-25T01:46:59.842 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-04-25T01:46:59.842 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-04-25T01:46:59.843 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-04-25T01:46:59.843 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-04-25T01:46:59.843 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-04-25T01:46:59.843 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-04-25T01:46:59.843 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-04-25T01:46:59.843 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-04-25T01:46:59.843 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-04-25T01:46:59.843 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-04-25T01:46:59.843 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-04-25T01:46:59.843 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-04-25T01:46:59.843 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-04-25T01:46:59.843 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-04-25T01:46:59.844 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-04-25T01:46:59.844 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-04-25T01:46:59.844 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-04-25T01:46:59.844 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-04-25T01:46:59.844 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-04-25T01:46:59.844 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-04-25T01:46:59.844 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-04-25T01:46:59.844 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-04-25T01:46:59.844 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-04-25T01:46:59.844 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-04-25T01:46:59.844 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-04-25T01:46:59.844 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-04-25T01:46:59.844 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-04-25T01:46:59.845 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-04-25T01:46:59.845 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-04-25T01:46:59.845 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-04-25T01:46:59.845 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-04-25T01:46:59.845 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-04-25T01:46:59.845 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-04-25T01:46:59.845 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-04-25T01:46:59.845 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-04-25T01:46:59.845 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-04-25T01:46:59.845 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-04-25T01:46:59.845 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-04-25T01:46:59.845 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-04-25T01:46:59.846 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-04-25T01:46:59.846 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-04-25T01:46:59.846 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-04-25T01:46:59.846 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-04-25T01:46:59.846 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-04-25T01:46:59.846 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-04-25T01:46:59.846 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-04-25T01:46:59.846 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-04-25T01:46:59.846 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-04-25T01:46:59.846 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-04-25T01:46:59.846 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-04-25T01:46:59.846 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-04-25T01:46:59.847 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-04-25T01:46:59.847 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-04-25T01:46:59.847 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-04-25T01:46:59.847 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-04-25T01:46:59.847 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-04-25T01:46:59.847 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-04-25T01:46:59.847 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-04-25T01:46:59.847 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-04-25T01:46:59.847 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-04-25T01:46:59.847 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-04-25T01:46:59.847 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-04-25T01:46:59.848 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-04-25T01:46:59.848 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-04-25T01:46:59.848 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-04-25T01:46:59.848 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-04-25T01:46:59.848 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-04-25T01:46:59.848 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-04-25T01:46:59.848 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-04-25T01:46:59.848 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-04-25T01:46:59.848 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-04-25T01:46:59.848 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-04-25T01:46:59.848 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-04-25T01:46:59.848 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-04-25T01:46:59.849 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-04-25T01:46:59.849 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-04-25T01:46:59.849 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-04-25T01:46:59.849 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-04-25T01:46:59.849 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-04-25T01:46:59.849 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-04-25T01:46:59.871 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-04-25T01:46:59.871 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-04-25T01:46:59.871 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-04-25T01:46:59.871 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-04-25T01:46:59.871 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-04-25T01:46:59.871 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-04-25T01:46:59.871 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-04-25T01:46:59.871 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-04-25T01:46:59.871 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-04-25T01:46:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-04-25T01:46:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-04-25T01:46:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-04-25T01:46:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-04-25T01:46:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-04-25T01:46:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-04-25T01:46:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-04-25T01:46:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-04-25T01:46:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-04-25T01:46:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-04-25T01:46:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-04-25T01:46:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-04-25T01:46:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-04-25T01:46:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-04-25T01:46:59.873 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-04-25T01:46:59.874 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-04-25T01:46:59.874 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-04-25T01:46:59.874 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-04-25T01:46:59.874 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-04-25T01:46:59.874 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-04-25T01:46:59.874 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-04-25T01:46:59.874 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-04-25T01:46:59.874 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-04-25T01:46:59.874 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-04-25T01:46:59.874 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-04-25T01:46:59.874 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-04-25T01:46:59.874 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-04-25T01:46:59.874 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-04-25T01:46:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-04-25T01:46:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-04-25T01:46:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-04-25T01:46:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-04-25T01:46:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-04-25T01:46:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-04-25T01:46:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-04-25T01:46:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-04-25T01:46:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-04-25T01:46:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-04-25T01:46:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-04-25T01:46:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-04-25T01:46:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-04-25T01:46:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-04-25T01:46:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-04-25T01:46:59.877 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-04-25T01:46:59.877 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-04-25T01:46:59.877 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-04-25T01:46:59.911 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-04-25T01:46:59.911 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-04-25T01:46:59.911 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-04-25T01:46:59.911 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-04-25T01:46:59.911 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-04-25T01:46:59.911 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-04-25T01:46:59.911 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-04-25T01:46:59.911 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-04-25T01:46:59.911 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-04-25T01:46:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-04-25T01:46:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-04-25T01:46:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-04-25T01:46:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-04-25T01:46:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-04-25T01:46:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-04-25T01:46:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-04-25T01:46:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-04-25T01:46:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-04-25T01:46:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-04-25T01:46:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-04-25T01:46:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-04-25T01:46:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-04-25T01:46:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-04-25T01:46:59.913 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-04-25T01:46:59.914 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-04-25T01:46:59.914 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-04-25T01:46:59.914 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-04-25T01:46:59.914 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-04-25T01:46:59.914 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-04-25T01:46:59.914 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-04-25T01:46:59.914 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-04-25T01:46:59.914 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-04-25T01:46:59.914 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-04-25T01:46:59.914 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-04-25T01:46:59.914 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-04-25T01:46:59.914 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-04-25T01:46:59.914 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-04-25T01:46:59.914 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-04-25T01:46:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-04-25T01:46:59.916 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-04-25T01:46:59.943 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-04-25T01:46:59.943 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-04-25T01:46:59.943 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-04-25T01:46:59.943 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-04-25T01:46:59.943 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-04-25T01:46:59.943 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-04-25T01:46:59.943 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-04-25T01:46:59.943 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-04-25T01:46:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-04-25T01:46:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-04-25T01:46:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-04-25T01:46:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-04-25T01:46:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-04-25T01:46:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-04-25T01:46:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-04-25T01:46:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-04-25T01:46:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-04-25T01:46:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-04-25T01:46:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-04-25T01:46:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-04-25T01:46:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-04-25T01:46:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-04-25T01:46:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-04-25T01:46:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-04-25T01:46:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-04-25T01:46:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-04-25T01:46:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-04-25T01:46:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-04-25T01:46:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-04-25T01:46:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-04-25T01:46:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-04-25T01:46:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-04-25T01:46:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-04-25T01:46:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-04-25T01:46:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-04-25T01:46:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-04-25T01:46:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-04-25T01:46:59.947 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-04-25T01:46:59.948 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-04-25T01:46:59.948 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-04-25T01:46:59.948 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-04-25T01:46:59.948 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-04-25T01:46:59.948 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-04-25T01:46:59.948 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-04-25T01:46:59.948 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-04-25T01:46:59.948 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-04-25T01:46:59.948 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-04-25T01:46:59.948 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-04-25T01:46:59.948 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-04-25T01:46:59.948 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-04-25T01:46:59.948 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-04-25T01:46:59.948 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-04-25T01:46:59.949 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-04-25T01:46:59.949 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-04-25T01:46:59.966 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-04-25T01:46:59.966 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-04-25T01:46:59.966 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-04-25T01:46:59.966 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-04-25T01:46:59.966 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-04-25T01:46:59.966 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-04-25T01:46:59.966 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-04-25T01:46:59.966 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-04-25T01:46:59.966 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-04-25T01:46:59.966 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-04-25T01:46:59.966 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-04-25T01:46:59.967 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-04-25T01:46:59.967 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-04-25T01:46:59.967 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-04-25T01:46:59.967 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-04-25T01:46:59.967 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-04-25T01:46:59.967 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-04-25T01:46:59.967 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-04-25T01:46:59.967 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-04-25T01:46:59.967 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-04-25T01:46:59.967 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-04-25T01:46:59.967 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-04-25T01:46:59.967 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-04-25T01:46:59.967 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-04-25T01:46:59.967 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-04-25T01:46:59.968 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-04-25T01:46:59.968 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-04-25T01:46:59.968 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-04-25T01:46:59.968 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-04-25T01:46:59.968 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-04-25T01:46:59.968 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-04-25T01:46:59.968 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-04-25T01:46:59.968 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-04-25T01:46:59.968 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-04-25T01:46:59.968 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-04-25T01:46:59.968 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-04-25T01:46:59.968 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-04-25T01:46:59.968 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-04-25T01:46:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-04-25T01:46:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-04-25T01:46:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-04-25T01:46:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-04-25T01:46:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-04-25T01:46:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-04-25T01:46:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-04-25T01:46:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-04-25T01:46:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-04-25T01:46:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-04-25T01:46:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-04-25T01:46:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-04-25T01:46:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-04-25T01:46:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-04-25T01:46:59.970 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-04-25T01:46:59.970 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-04-25T01:46:59.970 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-04-25T01:46:59.970 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-04-25T01:46:59.970 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-04-25T01:46:59.970 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-04-25T01:46:59.970 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-04-25T01:46:59.970 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-04-25T01:46:59.970 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-04-25T01:46:59.970 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-04-25T01:46:59.970 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-04-25T01:46:59.970 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-04-25T01:46:59.970 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-04-25T01:46:59.971 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-04-25T01:46:59.971 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-04-25T01:46:59.971 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-04-25T01:46:59.971 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-04-25T01:46:59.971 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-04-25T01:46:59.971 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-04-25T01:46:59.971 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-04-25T01:46:59.971 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-04-25T01:46:59.971 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-04-25T01:46:59.971 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-04-25T01:46:59.971 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-04-25T01:46:59.971 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-04-25T01:46:59.971 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-04-25T01:46:59.971 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-04-25T01:46:59.972 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-04-25T01:46:59.972 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-04-25T01:46:59.972 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-04-25T01:46:59.972 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-04-25T01:46:59.997 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-04-25T01:46:59.997 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-04-25T01:46:59.997 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-04-25T01:46:59.997 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-04-25T01:46:59.998 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-04-25T01:46:59.998 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-04-25T01:46:59.998 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-04-25T01:46:59.998 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-04-25T01:46:59.998 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-04-25T01:46:59.998 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-04-25T01:46:59.998 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-04-25T01:46:59.998 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-04-25T01:46:59.998 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-04-25T01:46:59.998 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-04-25T01:46:59.998 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-04-25T01:46:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-04-25T01:46:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-04-25T01:46:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-04-25T01:46:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-04-25T01:46:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-04-25T01:46:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-04-25T01:46:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-04-25T01:46:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-04-25T01:46:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-04-25T01:46:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-04-25T01:46:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-04-25T01:46:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-04-25T01:46:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-04-25T01:47:00.000 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-04-25T01:47:00.000 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-04-25T01:47:00.000 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-04-25T01:47:00.000 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-04-25T01:47:00.001 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-04-25T01:47:00.001 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-04-25T01:47:00.001 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-04-25T01:47:00.001 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-04-25T01:47:00.001 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-04-25T01:47:00.001 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-04-25T01:47:00.001 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-04-25T01:47:00.001 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-04-25T01:47:00.001 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-04-25T01:47:00.001 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-04-25T01:47:00.001 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-04-25T01:47:00.001 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-04-25T01:47:00.001 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-04-25T01:47:00.002 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-04-25T01:47:00.002 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-04-25T01:47:00.002 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-04-25T01:47:00.002 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-04-25T01:47:00.002 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-04-25T01:47:00.002 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-04-25T01:47:00.002 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-04-25T01:47:00.002 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-04-25T01:47:00.002 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-04-25T01:47:00.002 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-04-25T01:47:00.002 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-04-25T01:47:00.002 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-04-25T01:47:00.002 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-04-25T01:47:00.003 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-04-25T01:47:00.003 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-04-25T01:47:00.003 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-04-25T01:47:00.003 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-04-25T01:47:00.003 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-04-25T01:47:00.003 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-04-25T01:47:00.003 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-04-25T01:47:00.003 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-04-25T01:47:00.003 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-04-25T01:47:00.003 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-04-25T01:47:00.003 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-04-25T01:47:00.003 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-04-25T01:47:00.003 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-04-25T01:47:00.003 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-04-25T01:47:00.004 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-04-25T01:47:00.004 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-04-25T01:47:00.004 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-04-25T01:47:00.004 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-04-25T01:47:00.004 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-04-25T01:47:00.004 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-04-25T01:47:00.004 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-04-25T01:47:00.004 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-04-25T01:47:00.004 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-04-25T01:47:00.004 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-04-25T01:47:00.039 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-04-25T01:47:00.039 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-04-25T01:47:00.039 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-04-25T01:47:00.039 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-04-25T01:47:00.039 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-04-25T01:47:00.039 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-04-25T01:47:00.040 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-04-25T01:47:00.040 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-04-25T01:47:00.040 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-04-25T01:47:00.040 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-04-25T01:47:00.040 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-04-25T01:47:00.040 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-04-25T01:47:00.040 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-04-25T01:47:00.040 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-04-25T01:47:00.040 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-04-25T01:47:00.040 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-04-25T01:47:00.040 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-04-25T01:47:00.040 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-04-25T01:47:00.040 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-04-25T01:47:00.041 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-04-25T01:47:00.041 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-04-25T01:47:00.041 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-04-25T01:47:00.041 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-04-25T01:47:00.041 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-04-25T01:47:00.041 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-04-25T01:47:00.041 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-04-25T01:47:00.041 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-04-25T01:47:00.041 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-04-25T01:47:00.041 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-04-25T01:47:00.041 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-04-25T01:47:00.041 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-04-25T01:47:00.041 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-04-25T01:47:00.041 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-04-25T01:47:00.042 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-04-25T01:47:00.042 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-04-25T01:47:00.042 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-04-25T01:47:00.042 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-04-25T01:47:00.042 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-04-25T01:47:00.042 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-04-25T01:47:00.042 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-04-25T01:47:00.042 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-04-25T01:47:00.042 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-04-25T01:47:00.042 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-04-25T01:47:00.042 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-04-25T01:47:00.042 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-04-25T01:47:00.042 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-04-25T01:47:00.042 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-04-25T01:47:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-04-25T01:47:00.044 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-04-25T01:47:00.044 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-04-25T01:47:00.044 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-04-25T01:47:00.044 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-04-25T01:47:00.044 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-04-25T01:47:00.044 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-04-25T01:47:00.044 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-04-25T01:47:00.044 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-04-25T01:47:00.044 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-04-25T01:47:00.044 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-04-25T01:47:00.044 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-04-25T01:47:00.044 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-04-25T01:47:00.044 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-04-25T01:47:00.044 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-04-25T01:47:00.045 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-04-25T01:47:00.045 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-04-25T01:47:00.045 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-04-25T01:47:00.045 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-04-25T01:47:00.045 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-04-25T01:47:00.045 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-04-25T01:47:00.070 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-04-25T01:47:00.070 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-04-25T01:47:00.070 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-04-25T01:47:00.070 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-04-25T01:47:00.070 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-04-25T01:47:00.070 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-04-25T01:47:00.070 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-04-25T01:47:00.070 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-04-25T01:47:00.070 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-04-25T01:47:00.070 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-04-25T01:47:00.070 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-04-25T01:47:00.071 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-04-25T01:47:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-04-25T01:47:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-04-25T01:47:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-04-25T01:47:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-04-25T01:47:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-04-25T01:47:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-04-25T01:47:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-04-25T01:47:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-04-25T01:47:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-04-25T01:47:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-04-25T01:47:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-04-25T01:47:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-04-25T01:47:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-04-25T01:47:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-04-25T01:47:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-04-25T01:47:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-04-25T01:47:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-04-25T01:47:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-04-25T01:47:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-04-25T01:47:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-04-25T01:47:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-04-25T01:47:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-04-25T01:47:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-04-25T01:47:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-04-25T01:47:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-04-25T01:47:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-04-25T01:47:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-04-25T01:47:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-04-25T01:47:00.074 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-04-25T01:47:00.074 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-04-25T01:47:00.074 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-04-25T01:47:00.074 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-04-25T01:47:00.074 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-04-25T01:47:00.074 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-04-25T01:47:00.074 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-04-25T01:47:00.074 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-04-25T01:47:00.074 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-04-25T01:47:00.074 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-04-25T01:47:00.074 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-04-25T01:47:00.074 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-04-25T01:47:00.074 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-04-25T01:47:00.074 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-04-25T01:47:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-04-25T01:47:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-04-25T01:47:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-04-25T01:47:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-04-25T01:47:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-04-25T01:47:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-04-25T01:47:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-04-25T01:47:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-04-25T01:47:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-04-25T01:47:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-04-25T01:47:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-04-25T01:47:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-04-25T01:47:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-04-25T01:47:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-04-25T01:47:00.092 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-04-25T01:47:00.092 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-04-25T01:47:00.092 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-04-25T01:47:00.092 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-04-25T01:47:00.092 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-04-25T01:47:00.092 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-04-25T01:47:00.092 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-04-25T01:47:00.092 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-04-25T01:47:00.092 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-04-25T01:47:00.093 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-04-25T01:47:00.093 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-04-25T01:47:00.093 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-04-25T01:47:00.093 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-04-25T01:47:00.093 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-04-25T01:47:00.093 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-04-25T01:47:00.093 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-04-25T01:47:00.093 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-04-25T01:47:00.093 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-04-25T01:47:00.093 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-04-25T01:47:00.093 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-04-25T01:47:00.093 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-04-25T01:47:00.093 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-04-25T01:47:00.094 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-04-25T01:47:00.094 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-04-25T01:47:00.094 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-04-25T01:47:00.094 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-04-25T01:47:00.094 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-04-25T01:47:00.094 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-04-25T01:47:00.094 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-04-25T01:47:00.094 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-04-25T01:47:00.094 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-04-25T01:47:00.094 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-04-25T01:47:00.094 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-04-25T01:47:00.094 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-04-25T01:47:00.095 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-04-25T01:47:00.095 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-04-25T01:47:00.095 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-04-25T01:47:00.095 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-04-25T01:47:00.095 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-04-25T01:47:00.095 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-04-25T01:47:00.095 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-04-25T01:47:00.095 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-04-25T01:47:00.095 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-04-25T01:47:00.095 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-04-25T01:47:00.096 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-04-25T01:47:00.096 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-04-25T01:47:00.096 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-04-25T01:47:00.096 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-04-25T01:47:00.096 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-04-25T01:47:00.096 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-04-25T01:47:00.096 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-04-25T01:47:00.096 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-04-25T01:47:00.096 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-04-25T01:47:00.096 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-04-25T01:47:00.096 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-04-25T01:47:00.097 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-04-25T01:47:00.098 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-04-25T01:47:00.098 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-04-25T01:47:00.098 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-04-25T01:47:00.098 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-04-25T01:47:00.098 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-04-25T01:47:00.098 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-04-25T01:47:00.098 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-04-25T01:47:00.098 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-04-25T01:47:00.098 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-04-25T01:47:00.098 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-04-25T01:47:00.098 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-04-25T01:47:00.099 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-04-25T01:47:00.127 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-04-25T01:47:00.127 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-04-25T01:47:00.127 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-04-25T01:47:00.127 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-04-25T01:47:00.127 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-04-25T01:47:00.128 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-04-25T01:47:00.128 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-04-25T01:47:00.128 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-04-25T01:47:00.128 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-04-25T01:47:00.128 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-04-25T01:47:00.128 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-04-25T01:47:00.128 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-04-25T01:47:00.128 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-04-25T01:47:00.128 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-04-25T01:47:00.128 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-04-25T01:47:00.128 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-04-25T01:47:00.128 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-04-25T01:47:00.128 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-04-25T01:47:00.129 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-04-25T01:47:00.130 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-04-25T01:47:00.130 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-04-25T01:47:00.130 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-04-25T01:47:00.130 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-04-25T01:47:00.130 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-04-25T01:47:00.130 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-04-25T01:47:00.130 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-04-25T01:47:00.130 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-04-25T01:47:00.130 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-04-25T01:47:00.130 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-04-25T01:47:00.130 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-04-25T01:47:00.130 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-04-25T01:47:00.130 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-04-25T01:47:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-04-25T01:47:00.132 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-04-25T01:47:00.132 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-04-25T01:47:00.132 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-04-25T01:47:00.132 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-04-25T01:47:00.132 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-04-25T01:47:00.132 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-04-25T01:47:00.132 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-04-25T01:47:00.132 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-04-25T01:47:00.132 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-04-25T01:47:00.132 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-04-25T01:47:00.132 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-04-25T01:47:00.132 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-04-25T01:47:00.133 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-04-25T01:47:00.133 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-04-25T01:47:00.133 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-04-25T01:47:00.133 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-04-25T01:47:00.133 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-04-25T01:47:00.133 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-04-25T01:47:00.133 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-04-25T01:47:00.133 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-04-25T01:47:00.133 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-04-25T01:47:00.133 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-04-25T01:47:00.173 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-04-25T01:47:00.174 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-04-25T01:47:00.174 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-04-25T01:47:00.174 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-04-25T01:47:00.174 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-04-25T01:47:00.174 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-04-25T01:47:00.174 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-04-25T01:47:00.174 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-04-25T01:47:00.174 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-04-25T01:47:00.174 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-04-25T01:47:00.174 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-04-25T01:47:00.174 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-04-25T01:47:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-04-25T01:47:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-04-25T01:47:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-04-25T01:47:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-04-25T01:47:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-04-25T01:47:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-04-25T01:47:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-04-25T01:47:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-04-25T01:47:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-04-25T01:47:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-04-25T01:47:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-04-25T01:47:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-04-25T01:47:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-04-25T01:47:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-04-25T01:47:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-04-25T01:47:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-04-25T01:47:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-04-25T01:47:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-04-25T01:47:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-04-25T01:47:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-04-25T01:47:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-04-25T01:47:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-04-25T01:47:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-04-25T01:47:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-04-25T01:47:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-04-25T01:47:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-04-25T01:47:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-04-25T01:47:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-04-25T01:47:00.177 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-04-25T01:47:00.177 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-04-25T01:47:00.177 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-04-25T01:47:00.177 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-04-25T01:47:00.177 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-04-25T01:47:00.177 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-04-25T01:47:00.177 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-04-25T01:47:00.177 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-04-25T01:47:00.177 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-04-25T01:47:00.177 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-04-25T01:47:00.177 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-04-25T01:47:00.177 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-04-25T01:47:00.177 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-04-25T01:47:00.178 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-04-25T01:47:00.178 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-04-25T01:47:00.178 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-04-25T01:47:00.178 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-04-25T01:47:00.178 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-04-25T01:47:00.178 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-04-25T01:47:00.178 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-04-25T01:47:00.178 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-04-25T01:47:00.178 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-04-25T01:47:00.178 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-04-25T01:47:00.178 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-04-25T01:47:00.178 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-04-25T01:47:00.178 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-04-25T01:47:00.178 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-04-25T01:47:00.179 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-04-25T01:47:00.179 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-04-25T01:47:00.179 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-04-25T01:47:00.179 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-04-25T01:47:00.179 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-04-25T01:47:00.179 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-04-25T01:47:00.179 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-04-25T01:47:00.179 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-04-25T01:47:00.179 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-04-25T01:47:00.179 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-04-25T01:47:00.179 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-04-25T01:47:00.179 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-04-25T01:47:00.179 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-04-25T01:47:00.179 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-04-25T01:47:00.180 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-04-25T01:47:00.202 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-04-25T01:47:00.202 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-04-25T01:47:00.202 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-04-25T01:47:00.202 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-04-25T01:47:00.202 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-04-25T01:47:00.203 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-04-25T01:47:00.203 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-04-25T01:47:00.203 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-04-25T01:47:00.203 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-04-25T01:47:00.203 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-04-25T01:47:00.203 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-04-25T01:47:00.203 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-04-25T01:47:00.203 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-04-25T01:47:00.203 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-04-25T01:47:00.203 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-04-25T01:47:00.203 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-04-25T01:47:00.203 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-04-25T01:47:00.203 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-04-25T01:47:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-04-25T01:47:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-04-25T01:47:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-04-25T01:47:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-04-25T01:47:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-04-25T01:47:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-04-25T01:47:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-04-25T01:47:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-04-25T01:47:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-04-25T01:47:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-04-25T01:47:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-04-25T01:47:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-04-25T01:47:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-04-25T01:47:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-04-25T01:47:00.205 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-04-25T01:47:00.205 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-04-25T01:47:00.205 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-04-25T01:47:00.205 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-04-25T01:47:00.205 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-04-25T01:47:00.205 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-04-25T01:47:00.205 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-04-25T01:47:00.205 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-04-25T01:47:00.205 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-04-25T01:47:00.206 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-04-25T01:47:00.206 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-04-25T01:47:00.206 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-04-25T01:47:00.206 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-04-25T01:47:00.206 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-04-25T01:47:00.206 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-04-25T01:47:00.206 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-04-25T01:47:00.206 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-04-25T01:47:00.206 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-04-25T01:47:00.206 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-04-25T01:47:00.206 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-04-25T01:47:00.206 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-04-25T01:47:00.207 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-04-25T01:47:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-04-25T01:47:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-04-25T01:47:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-04-25T01:47:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-04-25T01:47:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-04-25T01:47:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-04-25T01:47:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-04-25T01:47:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-04-25T01:47:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-04-25T01:47:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-04-25T01:47:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-04-25T01:47:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-04-25T01:47:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-04-25T01:47:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-04-25T01:47:00.223 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-04-25T01:47:00.223 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-04-25T01:47:00.223 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-04-25T01:47:00.223 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-04-25T01:47:00.224 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-04-25T01:47:00.224 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-04-25T01:47:00.224 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-04-25T01:47:00.224 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-04-25T01:47:00.224 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-04-25T01:47:00.224 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-04-25T01:47:00.224 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-04-25T01:47:00.224 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-04-25T01:47:00.224 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-04-25T01:47:00.224 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-04-25T01:47:00.224 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-04-25T01:47:00.224 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-04-25T01:47:00.224 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-04-25T01:47:00.224 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-04-25T01:47:00.225 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-04-25T01:47:00.226 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-04-25T01:47:00.226 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-04-25T01:47:00.226 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-04-25T01:47:00.226 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-04-25T01:47:00.226 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-04-25T01:47:00.226 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-04-25T01:47:00.226 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-04-25T01:47:00.226 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-04-25T01:47:00.226 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-04-25T01:47:00.226 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-04-25T01:47:00.226 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-04-25T01:47:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-04-25T01:47:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-04-25T01:47:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-04-25T01:47:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-04-25T01:47:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-04-25T01:47:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-04-25T01:47:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-04-25T01:47:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-04-25T01:47:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-04-25T01:47:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-04-25T01:47:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-04-25T01:47:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-04-25T01:47:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-04-25T01:47:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-04-25T01:47:00.228 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-04-25T01:47:00.228 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-04-25T01:47:00.228 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-04-25T01:47:00.228 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-04-25T01:47:00.228 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-04-25T01:47:00.228 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-04-25T01:47:00.228 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-04-25T01:47:00.228 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-04-25T01:47:00.228 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-04-25T01:47:00.228 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-04-25T01:47:00.229 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-04-25T01:47:00.229 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-04-25T01:47:00.229 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-04-25T01:47:00.229 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-04-25T01:47:00.229 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-04-25T01:47:00.229 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-04-25T01:47:00.229 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-04-25T01:47:00.229 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-04-25T01:47:00.229 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-04-25T01:47:00.229 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-04-25T01:47:00.229 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-04-25T01:47:00.230 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-04-25T01:47:00.230 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-04-25T01:47:00.230 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-04-25T01:47:00.230 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-04-25T01:47:00.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:00 smithi078 ceph-mon[29469]: pgmap v405: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 731 op/s 2024-04-25T01:47:00.265 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-04-25T01:47:00.265 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-04-25T01:47:00.265 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-04-25T01:47:00.265 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-04-25T01:47:00.265 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-04-25T01:47:00.265 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-04-25T01:47:00.266 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-04-25T01:47:00.266 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-04-25T01:47:00.266 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-04-25T01:47:00.266 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-04-25T01:47:00.266 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-04-25T01:47:00.266 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-04-25T01:47:00.266 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-04-25T01:47:00.267 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-04-25T01:47:00.267 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-04-25T01:47:00.267 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-04-25T01:47:00.267 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-04-25T01:47:00.267 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-04-25T01:47:00.267 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-04-25T01:47:00.267 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-04-25T01:47:00.267 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-04-25T01:47:00.268 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-04-25T01:47:00.268 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-04-25T01:47:00.268 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-04-25T01:47:00.268 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-04-25T01:47:00.268 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-04-25T01:47:00.268 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-04-25T01:47:00.268 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-04-25T01:47:00.268 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-04-25T01:47:00.269 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-04-25T01:47:00.269 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-04-25T01:47:00.269 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-04-25T01:47:00.269 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-04-25T01:47:00.269 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-04-25T01:47:00.269 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-04-25T01:47:00.270 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-04-25T01:47:00.270 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-04-25T01:47:00.270 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-04-25T01:47:00.270 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-04-25T01:47:00.270 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-04-25T01:47:00.270 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-04-25T01:47:00.270 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-04-25T01:47:00.271 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-04-25T01:47:00.271 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-04-25T01:47:00.271 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-04-25T01:47:00.271 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-04-25T01:47:00.271 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-04-25T01:47:00.271 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-04-25T01:47:00.271 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-04-25T01:47:00.271 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-04-25T01:47:00.271 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-04-25T01:47:00.272 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-04-25T01:47:00.272 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-04-25T01:47:00.272 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-04-25T01:47:00.272 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-04-25T01:47:00.272 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-04-25T01:47:00.272 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-04-25T01:47:00.272 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-04-25T01:47:00.272 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-04-25T01:47:00.272 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-04-25T01:47:00.272 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-04-25T01:47:00.273 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-04-25T01:47:00.273 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-04-25T01:47:00.273 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-04-25T01:47:00.273 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-04-25T01:47:00.273 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-04-25T01:47:00.273 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-04-25T01:47:00.273 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-04-25T01:47:00.273 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-04-25T01:47:00.273 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-04-25T01:47:00.273 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-04-25T01:47:00.274 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-04-25T01:47:00.274 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-04-25T01:47:00.274 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-04-25T01:47:00.274 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-04-25T01:47:00.274 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-04-25T01:47:00.274 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-04-25T01:47:00.274 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-04-25T01:47:00.275 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-04-25T01:47:00.275 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-04-25T01:47:00.275 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-04-25T01:47:00.275 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-04-25T01:47:00.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:00 smithi152 ceph-mon[32465]: pgmap v405: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 731 op/s 2024-04-25T01:47:00.311 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-04-25T01:47:00.311 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-04-25T01:47:00.311 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-04-25T01:47:00.311 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-04-25T01:47:00.311 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-04-25T01:47:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-04-25T01:47:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-04-25T01:47:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-04-25T01:47:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-04-25T01:47:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-04-25T01:47:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-04-25T01:47:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-04-25T01:47:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-04-25T01:47:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-04-25T01:47:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-04-25T01:47:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-04-25T01:47:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-04-25T01:47:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-04-25T01:47:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-04-25T01:47:00.313 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-04-25T01:47:00.313 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-04-25T01:47:00.313 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-04-25T01:47:00.313 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-04-25T01:47:00.313 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-04-25T01:47:00.313 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-04-25T01:47:00.314 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-04-25T01:47:00.314 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-04-25T01:47:00.314 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-04-25T01:47:00.314 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-04-25T01:47:00.314 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-04-25T01:47:00.314 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-04-25T01:47:00.314 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-04-25T01:47:00.315 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-04-25T01:47:00.315 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-04-25T01:47:00.315 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-04-25T01:47:00.315 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-04-25T01:47:00.315 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-04-25T01:47:00.315 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-04-25T01:47:00.316 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-04-25T01:47:00.316 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-04-25T01:47:00.316 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-04-25T01:47:00.316 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-04-25T01:47:00.316 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-04-25T01:47:00.316 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-04-25T01:47:00.316 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-04-25T01:47:00.317 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-04-25T01:47:00.317 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-04-25T01:47:00.317 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-04-25T01:47:00.317 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-04-25T01:47:00.317 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-04-25T01:47:00.317 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-04-25T01:47:00.317 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-04-25T01:47:00.318 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-04-25T01:47:00.318 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-04-25T01:47:00.318 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-04-25T01:47:00.318 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-04-25T01:47:00.318 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-04-25T01:47:00.318 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-04-25T01:47:00.319 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-04-25T01:47:00.319 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-04-25T01:47:00.319 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-04-25T01:47:00.319 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-04-25T01:47:00.319 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-04-25T01:47:00.319 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-04-25T01:47:00.320 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-04-25T01:47:00.320 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-04-25T01:47:00.320 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-04-25T01:47:00.320 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-04-25T01:47:00.320 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-04-25T01:47:00.320 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-04-25T01:47:00.320 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-04-25T01:47:00.321 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-04-25T01:47:00.321 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-04-25T01:47:00.321 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-04-25T01:47:00.321 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-04-25T01:47:00.321 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-04-25T01:47:00.321 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-04-25T01:47:00.321 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-04-25T01:47:00.322 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-04-25T01:47:00.322 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-04-25T01:47:00.322 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-04-25T01:47:00.322 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-04-25T01:47:00.322 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-04-25T01:47:00.335 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-04-25T01:47:00.335 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-04-25T01:47:00.335 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-04-25T01:47:00.335 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-04-25T01:47:00.335 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-04-25T01:47:00.335 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-04-25T01:47:00.335 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-04-25T01:47:00.335 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-04-25T01:47:00.335 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-04-25T01:47:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-04-25T01:47:00.337 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-04-25T01:47:00.337 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-04-25T01:47:00.337 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-04-25T01:47:00.337 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-04-25T01:47:00.337 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-04-25T01:47:00.337 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-04-25T01:47:00.337 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-04-25T01:47:00.337 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-04-25T01:47:00.337 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-04-25T01:47:00.337 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-04-25T01:47:00.337 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-04-25T01:47:00.337 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-04-25T01:47:00.337 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-04-25T01:47:00.338 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-04-25T01:47:00.338 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-04-25T01:47:00.338 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-04-25T01:47:00.338 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-04-25T01:47:00.338 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-04-25T01:47:00.338 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-04-25T01:47:00.338 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-04-25T01:47:00.338 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-04-25T01:47:00.338 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-04-25T01:47:00.338 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-04-25T01:47:00.338 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-04-25T01:47:00.338 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-04-25T01:47:00.338 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-04-25T01:47:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-04-25T01:47:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-04-25T01:47:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-04-25T01:47:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-04-25T01:47:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-04-25T01:47:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-04-25T01:47:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-04-25T01:47:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-04-25T01:47:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-04-25T01:47:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-04-25T01:47:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-04-25T01:47:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-04-25T01:47:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-04-25T01:47:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-04-25T01:47:00.340 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-04-25T01:47:00.340 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-04-25T01:47:00.340 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-04-25T01:47:00.340 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-04-25T01:47:00.340 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-04-25T01:47:00.340 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-04-25T01:47:00.340 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-04-25T01:47:00.340 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-04-25T01:47:00.340 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-04-25T01:47:00.340 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-04-25T01:47:00.340 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-04-25T01:47:00.340 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-04-25T01:47:00.340 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-04-25T01:47:00.341 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-04-25T01:47:00.341 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-04-25T01:47:00.341 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-04-25T01:47:00.341 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-04-25T01:47:00.341 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-04-25T01:47:00.356 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-04-25T01:47:00.356 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-04-25T01:47:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-04-25T01:47:00.358 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-04-25T01:47:00.358 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-04-25T01:47:00.358 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-04-25T01:47:00.358 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-04-25T01:47:00.358 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-04-25T01:47:00.358 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-04-25T01:47:00.358 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-04-25T01:47:00.358 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-04-25T01:47:00.358 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-04-25T01:47:00.358 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-04-25T01:47:00.358 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-04-25T01:47:00.359 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-04-25T01:47:00.359 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-04-25T01:47:00.359 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-04-25T01:47:00.359 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-04-25T01:47:00.359 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-04-25T01:47:00.359 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-04-25T01:47:00.359 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-04-25T01:47:00.359 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-04-25T01:47:00.359 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-04-25T01:47:00.359 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-04-25T01:47:00.359 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-04-25T01:47:00.359 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-04-25T01:47:00.359 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-04-25T01:47:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-04-25T01:47:00.361 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-04-25T01:47:00.362 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-04-25T01:47:00.362 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-04-25T01:47:00.362 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-04-25T01:47:00.362 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-04-25T01:47:00.362 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-04-25T01:47:00.362 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-04-25T01:47:00.362 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-04-25T01:47:00.362 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-04-25T01:47:00.362 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-04-25T01:47:00.362 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-04-25T01:47:00.362 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-04-25T01:47:00.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:00 smithi196 ceph-mon[32978]: pgmap v405: 129 pgs: 129 active+clean; 1.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 731 op/s 2024-04-25T01:47:00.393 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-04-25T01:47:00.393 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-04-25T01:47:00.393 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-04-25T01:47:00.393 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-04-25T01:47:00.393 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-04-25T01:47:00.393 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-04-25T01:47:00.393 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-04-25T01:47:00.394 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-04-25T01:47:00.394 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-04-25T01:47:00.394 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-04-25T01:47:00.394 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-04-25T01:47:00.394 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-04-25T01:47:00.394 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-04-25T01:47:00.395 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-04-25T01:47:00.395 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-04-25T01:47:00.395 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-04-25T01:47:00.395 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-04-25T01:47:00.395 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-04-25T01:47:00.395 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-04-25T01:47:00.396 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-04-25T01:47:00.396 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-04-25T01:47:00.396 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-04-25T01:47:00.396 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-04-25T01:47:00.396 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-04-25T01:47:00.396 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-04-25T01:47:00.396 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-04-25T01:47:00.397 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-04-25T01:47:00.397 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-04-25T01:47:00.397 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-04-25T01:47:00.397 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-04-25T01:47:00.397 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-04-25T01:47:00.397 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-04-25T01:47:00.398 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-04-25T01:47:00.398 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-04-25T01:47:00.398 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-04-25T01:47:00.398 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-04-25T01:47:00.398 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-04-25T01:47:00.399 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-04-25T01:47:00.399 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-04-25T01:47:00.399 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-04-25T01:47:00.399 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-04-25T01:47:00.399 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-04-25T01:47:00.399 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-04-25T01:47:00.400 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-04-25T01:47:00.400 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-04-25T01:47:00.400 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-04-25T01:47:00.400 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-04-25T01:47:00.400 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-04-25T01:47:00.400 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-04-25T01:47:00.400 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-04-25T01:47:00.401 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-04-25T01:47:00.401 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-04-25T01:47:00.401 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-04-25T01:47:00.401 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-04-25T01:47:00.401 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-04-25T01:47:00.402 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-04-25T01:47:00.402 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-04-25T01:47:00.402 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-04-25T01:47:00.402 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-04-25T01:47:00.402 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-04-25T01:47:00.402 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-04-25T01:47:00.403 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-04-25T01:47:00.403 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-04-25T01:47:00.403 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-04-25T01:47:00.403 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-04-25T01:47:00.403 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-04-25T01:47:00.404 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-04-25T01:47:00.404 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-04-25T01:47:00.404 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-04-25T01:47:00.404 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-04-25T01:47:00.404 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-04-25T01:47:00.404 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-04-25T01:47:00.404 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-04-25T01:47:00.405 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-04-25T01:47:00.405 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-04-25T01:47:00.405 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-04-25T01:47:00.405 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-04-25T01:47:00.405 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-04-25T01:47:00.405 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-04-25T01:47:00.406 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-04-25T01:47:00.406 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-04-25T01:47:00.406 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-04-25T01:47:00.406 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-04-25T01:47:00.440 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-04-25T01:47:00.440 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-04-25T01:47:00.440 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-04-25T01:47:00.440 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-04-25T01:47:00.440 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-04-25T01:47:00.440 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-04-25T01:47:00.440 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-04-25T01:47:00.440 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-04-25T01:47:00.440 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-04-25T01:47:00.441 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-04-25T01:47:00.441 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-04-25T01:47:00.441 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-04-25T01:47:00.441 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-04-25T01:47:00.441 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-04-25T01:47:00.441 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-04-25T01:47:00.441 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-04-25T01:47:00.441 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-04-25T01:47:00.441 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-04-25T01:47:00.441 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-04-25T01:47:00.441 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-04-25T01:47:00.441 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-04-25T01:47:00.442 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-04-25T01:47:00.442 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-04-25T01:47:00.442 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-04-25T01:47:00.442 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-04-25T01:47:00.442 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-04-25T01:47:00.442 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-04-25T01:47:00.442 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-04-25T01:47:00.442 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-04-25T01:47:00.442 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-04-25T01:47:00.442 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-04-25T01:47:00.442 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-04-25T01:47:00.442 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-04-25T01:47:00.442 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-04-25T01:47:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-04-25T01:47:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-04-25T01:47:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-04-25T01:47:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-04-25T01:47:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-04-25T01:47:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-04-25T01:47:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-04-25T01:47:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-04-25T01:47:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-04-25T01:47:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-04-25T01:47:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-04-25T01:47:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-04-25T01:47:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-04-25T01:47:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-04-25T01:47:00.444 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-04-25T01:47:00.444 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-04-25T01:47:00.444 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-04-25T01:47:00.444 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-04-25T01:47:00.444 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-04-25T01:47:00.444 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-04-25T01:47:00.444 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-04-25T01:47:00.444 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-04-25T01:47:00.445 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-04-25T01:47:00.445 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-04-25T01:47:00.445 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-04-25T01:47:00.445 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-04-25T01:47:00.445 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-04-25T01:47:00.445 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-04-25T01:47:00.445 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-04-25T01:47:00.446 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-04-25T01:47:00.446 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-04-25T01:47:00.446 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-04-25T01:47:00.446 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-04-25T01:47:00.446 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-04-25T01:47:00.446 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-04-25T01:47:00.446 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-04-25T01:47:00.446 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-04-25T01:47:00.446 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-04-25T01:47:00.446 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-04-25T01:47:00.446 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-04-25T01:47:00.447 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-04-25T01:47:00.447 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-04-25T01:47:00.447 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-04-25T01:47:00.447 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-04-25T01:47:00.447 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-04-25T01:47:00.447 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-04-25T01:47:00.448 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-04-25T01:47:00.448 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-04-25T01:47:00.466 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-04-25T01:47:00.467 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-04-25T01:47:00.467 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-04-25T01:47:00.467 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-04-25T01:47:00.467 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-04-25T01:47:00.467 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-04-25T01:47:00.467 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-04-25T01:47:00.467 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-04-25T01:47:00.467 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-04-25T01:47:00.467 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-04-25T01:47:00.468 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-04-25T01:47:00.468 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-04-25T01:47:00.468 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-04-25T01:47:00.468 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-04-25T01:47:00.468 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-04-25T01:47:00.468 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-04-25T01:47:00.468 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-04-25T01:47:00.468 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-04-25T01:47:00.468 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-04-25T01:47:00.468 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-04-25T01:47:00.468 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-04-25T01:47:00.469 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-04-25T01:47:00.469 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-04-25T01:47:00.469 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-04-25T01:47:00.469 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-04-25T01:47:00.469 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-04-25T01:47:00.469 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-04-25T01:47:00.469 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-04-25T01:47:00.470 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-04-25T01:47:00.470 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-04-25T01:47:00.470 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-04-25T01:47:00.470 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-04-25T01:47:00.470 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-04-25T01:47:00.470 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-04-25T01:47:00.471 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-04-25T01:47:00.471 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-04-25T01:47:00.471 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-04-25T01:47:00.471 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-04-25T01:47:00.471 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-04-25T01:47:00.471 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-04-25T01:47:00.471 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-04-25T01:47:00.472 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-04-25T01:47:00.472 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-04-25T01:47:00.472 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-04-25T01:47:00.472 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-04-25T01:47:00.472 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-04-25T01:47:00.472 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-04-25T01:47:00.472 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-04-25T01:47:00.472 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-04-25T01:47:00.472 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-04-25T01:47:00.472 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-04-25T01:47:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-04-25T01:47:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-04-25T01:47:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-04-25T01:47:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-04-25T01:47:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-04-25T01:47:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-04-25T01:47:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-04-25T01:47:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-04-25T01:47:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-04-25T01:47:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-04-25T01:47:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-04-25T01:47:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-04-25T01:47:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-04-25T01:47:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-04-25T01:47:00.474 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-04-25T01:47:00.474 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-04-25T01:47:00.474 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-04-25T01:47:00.474 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-04-25T01:47:00.474 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-04-25T01:47:00.474 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-04-25T01:47:00.474 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-04-25T01:47:00.474 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-04-25T01:47:00.474 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-04-25T01:47:00.474 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-04-25T01:47:00.474 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-04-25T01:47:00.474 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-04-25T01:47:00.474 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-04-25T01:47:00.475 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-04-25T01:47:00.475 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-04-25T01:47:00.475 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-04-25T01:47:00.475 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-04-25T01:47:00.475 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-04-25T01:47:00.490 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-04-25T01:47:00.490 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-04-25T01:47:00.490 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-04-25T01:47:00.491 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-04-25T01:47:00.491 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-04-25T01:47:00.491 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-04-25T01:47:00.491 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-04-25T01:47:00.491 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-04-25T01:47:00.491 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-04-25T01:47:00.491 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-04-25T01:47:00.491 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-04-25T01:47:00.491 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-04-25T01:47:00.491 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-04-25T01:47:00.491 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-04-25T01:47:00.491 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-04-25T01:47:00.492 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-04-25T01:47:00.492 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-04-25T01:47:00.492 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-04-25T01:47:00.492 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-04-25T01:47:00.492 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-04-25T01:47:00.492 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-04-25T01:47:00.492 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-04-25T01:47:00.492 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-04-25T01:47:00.492 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-04-25T01:47:00.493 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-04-25T01:47:00.493 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-04-25T01:47:00.493 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-04-25T01:47:00.493 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-04-25T01:47:00.493 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-04-25T01:47:00.493 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-04-25T01:47:00.493 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-04-25T01:47:00.493 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-04-25T01:47:00.493 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-04-25T01:47:00.493 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-04-25T01:47:00.493 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-04-25T01:47:00.494 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-04-25T01:47:00.494 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-04-25T01:47:00.494 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-04-25T01:47:00.494 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-04-25T01:47:00.494 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-04-25T01:47:00.494 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-04-25T01:47:00.494 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-04-25T01:47:00.494 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-04-25T01:47:00.494 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-04-25T01:47:00.494 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-04-25T01:47:00.494 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-04-25T01:47:00.494 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-04-25T01:47:00.494 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-04-25T01:47:00.494 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-04-25T01:47:00.495 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-04-25T01:47:00.495 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-04-25T01:47:00.495 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-04-25T01:47:00.495 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-04-25T01:47:00.495 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-04-25T01:47:00.495 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-04-25T01:47:00.495 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-04-25T01:47:00.495 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-04-25T01:47:00.495 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-04-25T01:47:00.495 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-04-25T01:47:00.495 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-04-25T01:47:00.495 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-04-25T01:47:00.495 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-04-25T01:47:00.496 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-04-25T01:47:00.496 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-04-25T01:47:00.496 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-04-25T01:47:00.496 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-04-25T01:47:00.496 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-04-25T01:47:00.496 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-04-25T01:47:00.496 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-04-25T01:47:00.496 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-04-25T01:47:00.496 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-04-25T01:47:00.496 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-04-25T01:47:00.496 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-04-25T01:47:00.496 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-04-25T01:47:00.496 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-04-25T01:47:00.496 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-04-25T01:47:00.497 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-04-25T01:47:00.497 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-04-25T01:47:00.497 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-04-25T01:47:00.497 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-04-25T01:47:00.497 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-04-25T01:47:00.497 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-04-25T01:47:00.527 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-04-25T01:47:00.528 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-04-25T01:47:00.528 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-04-25T01:47:00.528 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-04-25T01:47:00.528 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-04-25T01:47:00.528 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-04-25T01:47:00.528 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-04-25T01:47:00.528 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-04-25T01:47:00.528 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-04-25T01:47:00.528 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-04-25T01:47:00.528 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-04-25T01:47:00.528 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-04-25T01:47:00.528 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-04-25T01:47:00.528 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-04-25T01:47:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-04-25T01:47:00.530 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-04-25T01:47:00.530 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-04-25T01:47:00.530 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-04-25T01:47:00.530 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-04-25T01:47:00.530 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-04-25T01:47:00.530 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-04-25T01:47:00.530 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-04-25T01:47:00.530 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-04-25T01:47:00.530 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-04-25T01:47:00.530 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-04-25T01:47:00.530 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-04-25T01:47:00.530 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-04-25T01:47:00.530 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-04-25T01:47:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-04-25T01:47:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-04-25T01:47:00.533 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-04-25T01:47:00.533 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-04-25T01:47:00.533 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-04-25T01:47:00.533 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-04-25T01:47:00.533 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-04-25T01:47:00.533 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-04-25T01:47:00.533 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-04-25T01:47:00.533 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-04-25T01:47:00.533 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-04-25T01:47:00.533 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-04-25T01:47:00.566 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-04-25T01:47:00.566 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-04-25T01:47:00.566 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-04-25T01:47:00.566 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-04-25T01:47:00.566 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-04-25T01:47:00.566 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-04-25T01:47:00.566 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-04-25T01:47:00.567 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-04-25T01:47:00.567 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-04-25T01:47:00.567 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-04-25T01:47:00.567 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-04-25T01:47:00.567 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-04-25T01:47:00.567 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-04-25T01:47:00.567 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-04-25T01:47:00.567 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-04-25T01:47:00.567 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-04-25T01:47:00.567 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-04-25T01:47:00.567 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-04-25T01:47:00.567 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-04-25T01:47:00.567 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-04-25T01:47:00.567 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-04-25T01:47:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-04-25T01:47:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-04-25T01:47:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-04-25T01:47:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-04-25T01:47:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-04-25T01:47:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-04-25T01:47:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-04-25T01:47:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-04-25T01:47:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-04-25T01:47:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-04-25T01:47:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-04-25T01:47:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-04-25T01:47:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-04-25T01:47:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-04-25T01:47:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-04-25T01:47:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-04-25T01:47:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-04-25T01:47:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-04-25T01:47:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-04-25T01:47:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-04-25T01:47:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-04-25T01:47:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-04-25T01:47:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-04-25T01:47:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-04-25T01:47:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-04-25T01:47:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-04-25T01:47:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-04-25T01:47:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-04-25T01:47:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-04-25T01:47:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-04-25T01:47:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-04-25T01:47:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-04-25T01:47:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-04-25T01:47:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-04-25T01:47:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-04-25T01:47:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-04-25T01:47:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-04-25T01:47:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-04-25T01:47:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-04-25T01:47:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-04-25T01:47:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-04-25T01:47:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-04-25T01:47:00.571 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-04-25T01:47:00.572 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-04-25T01:47:00.572 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-04-25T01:47:00.572 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-04-25T01:47:00.572 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-04-25T01:47:00.587 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-04-25T01:47:00.588 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-04-25T01:47:00.588 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-04-25T01:47:00.588 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-04-25T01:47:00.588 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-04-25T01:47:00.588 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-04-25T01:47:00.588 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-04-25T01:47:00.588 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-04-25T01:47:00.588 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-04-25T01:47:00.588 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-04-25T01:47:00.589 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-04-25T01:47:00.589 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-04-25T01:47:00.589 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-04-25T01:47:00.589 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-04-25T01:47:00.589 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-04-25T01:47:00.589 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-04-25T01:47:00.589 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-04-25T01:47:00.589 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-04-25T01:47:00.589 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-04-25T01:47:00.589 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-04-25T01:47:00.589 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-04-25T01:47:00.589 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-04-25T01:47:00.590 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-04-25T01:47:00.590 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-04-25T01:47:00.590 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-04-25T01:47:00.590 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-04-25T01:47:00.590 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-04-25T01:47:00.590 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-04-25T01:47:00.590 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-04-25T01:47:00.590 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-04-25T01:47:00.590 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-04-25T01:47:00.590 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-04-25T01:47:00.590 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-04-25T01:47:00.591 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-04-25T01:47:00.591 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-04-25T01:47:00.591 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-04-25T01:47:00.591 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-04-25T01:47:00.591 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-04-25T01:47:00.591 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-04-25T01:47:00.591 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-04-25T01:47:00.591 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-04-25T01:47:00.591 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-04-25T01:47:00.591 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-04-25T01:47:00.591 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-04-25T01:47:00.591 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-04-25T01:47:00.591 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-04-25T01:47:00.592 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-04-25T01:47:00.592 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-04-25T01:47:00.592 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-04-25T01:47:00.592 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-04-25T01:47:00.592 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-04-25T01:47:00.592 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-04-25T01:47:00.592 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-04-25T01:47:00.592 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-04-25T01:47:00.592 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-04-25T01:47:00.592 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-04-25T01:47:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-04-25T01:47:00.594 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-04-25T01:47:00.594 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-04-25T01:47:00.594 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-04-25T01:47:00.594 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-04-25T01:47:00.594 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-04-25T01:47:00.594 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-04-25T01:47:00.594 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-04-25T01:47:00.594 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-04-25T01:47:00.594 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-04-25T01:47:00.594 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-04-25T01:47:00.595 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-04-25T01:47:00.922 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-04-25T01:47:00.922 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-04-25T01:47:00.923 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-04-25T01:47:00.923 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-04-25T01:47:00.923 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-04-25T01:47:00.923 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-04-25T01:47:00.923 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-04-25T01:47:00.923 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-04-25T01:47:00.923 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-04-25T01:47:00.923 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-04-25T01:47:00.923 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-04-25T01:47:00.923 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-04-25T01:47:00.923 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-04-25T01:47:00.924 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-04-25T01:47:00.924 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-04-25T01:47:00.924 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-04-25T01:47:00.924 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-04-25T01:47:00.924 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-04-25T01:47:00.924 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-04-25T01:47:00.924 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-04-25T01:47:00.924 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-04-25T01:47:00.924 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-04-25T01:47:00.924 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-04-25T01:47:00.924 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-04-25T01:47:00.924 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-04-25T01:47:00.924 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-04-25T01:47:00.925 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-04-25T01:47:00.925 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-04-25T01:47:00.925 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-04-25T01:47:00.925 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-04-25T01:47:00.925 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-04-25T01:47:00.925 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-04-25T01:47:00.925 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-04-25T01:47:00.925 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-04-25T01:47:00.925 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-04-25T01:47:00.925 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-04-25T01:47:00.925 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-04-25T01:47:00.925 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-04-25T01:47:00.926 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-04-25T01:47:00.926 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-04-25T01:47:00.926 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-04-25T01:47:00.926 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-04-25T01:47:00.926 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-04-25T01:47:00.926 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-04-25T01:47:00.926 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-04-25T01:47:00.926 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-04-25T01:47:00.926 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-04-25T01:47:00.926 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-04-25T01:47:00.927 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-04-25T01:47:00.927 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-04-25T01:47:00.927 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-04-25T01:47:00.927 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-04-25T01:47:00.927 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-04-25T01:47:00.927 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-04-25T01:47:00.927 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-04-25T01:47:00.927 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-04-25T01:47:00.927 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-04-25T01:47:00.927 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-04-25T01:47:00.927 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-04-25T01:47:00.927 INFO:tasks.workunit.client.0.smithi078.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-04-25T01:47:00.927 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-04-25T01:47:00.928 INFO:tasks.workunit.client.0.smithi078.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-04-25T01:47:00.928 INFO:tasks.workunit.client.0.smithi078.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-04-25T01:47:00.928 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-04-25T01:47:00.928 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-04-25T01:47:00.928 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-04-25T01:47:00.928 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-04-25T01:47:00.928 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-04-25T01:47:00.928 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-04-25T01:47:00.928 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-04-25T01:47:00.928 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-04-25T01:47:00.928 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-04-25T01:47:00.929 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-04-25T01:47:00.929 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-04-25T01:47:00.929 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-04-25T01:47:00.929 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-04-25T01:47:00.929 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-04-25T01:47:00.929 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-04-25T01:47:00.929 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-04-25T01:47:00.929 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-04-25T01:47:00.929 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-04-25T01:47:00.929 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-04-25T01:47:00.929 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-04-25T01:47:00.929 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-04-25T01:47:02.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-04-25T01:47:02.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-04-25T01:47:02.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-04-25T01:47:02.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-04-25T01:47:02.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-04-25T01:47:02.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-04-25T01:47:02.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-04-25T01:47:02.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-04-25T01:47:02.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-04-25T01:47:02.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-04-25T01:47:02.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-04-25T01:47:02.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-04-25T01:47:02.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-04-25T01:47:02.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-04-25T01:47:02.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-04-25T01:47:02.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-04-25T01:47:02.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-04-25T01:47:02.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-04-25T01:47:02.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-04-25T01:47:02.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-04-25T01:47:02.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-04-25T01:47:02.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-04-25T01:47:02.175 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-04-25T01:47:02.176 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-04-25T01:47:02.177 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-04-25T01:47:02.177 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-04-25T01:47:02.177 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-04-25T01:47:02.177 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-04-25T01:47:02.177 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-04-25T01:47:02.177 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-04-25T01:47:02.177 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-04-25T01:47:02.177 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-04-25T01:47:02.177 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-04-25T01:47:02.177 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-04-25T01:47:02.177 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-04-25T01:47:02.177 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-04-25T01:47:02.177 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-04-25T01:47:02.177 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-04-25T01:47:02.178 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-04-25T01:47:02.178 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-04-25T01:47:02.178 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-04-25T01:47:02.178 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-04-25T01:47:02.178 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-04-25T01:47:02.178 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-04-25T01:47:02.178 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-04-25T01:47:02.178 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-04-25T01:47:02.178 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-04-25T01:47:02.178 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-04-25T01:47:02.178 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-04-25T01:47:02.178 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-04-25T01:47:02.178 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-04-25T01:47:02.178 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-04-25T01:47:02.179 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-04-25T01:47:02.179 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-04-25T01:47:02.179 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-04-25T01:47:02.179 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-04-25T01:47:02.179 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-04-25T01:47:02.179 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-04-25T01:47:02.301 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:02 smithi152 ceph-mon[32465]: pgmap v406: 129 pgs: 129 active+clean; 1.2 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.4 MiB/s wr, 1.21k op/s 2024-04-25T01:47:02.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:02 smithi196 ceph-mon[32978]: pgmap v406: 129 pgs: 129 active+clean; 1.2 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.4 MiB/s wr, 1.21k op/s 2024-04-25T01:47:02.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:02 smithi078 ceph-mon[29469]: pgmap v406: 129 pgs: 129 active+clean; 1.2 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.4 MiB/s wr, 1.21k op/s 2024-04-25T01:47:03.003 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-04-25T01:47:03.003 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-04-25T01:47:03.003 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-04-25T01:47:03.003 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-04-25T01:47:03.003 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-04-25T01:47:03.003 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-04-25T01:47:03.003 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-04-25T01:47:03.003 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-04-25T01:47:03.003 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-04-25T01:47:03.003 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-04-25T01:47:03.003 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-04-25T01:47:03.004 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-04-25T01:47:03.004 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-04-25T01:47:03.004 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-04-25T01:47:03.004 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-04-25T01:47:03.004 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-04-25T01:47:03.004 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-04-25T01:47:03.004 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-04-25T01:47:03.004 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-04-25T01:47:03.004 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-04-25T01:47:03.004 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-04-25T01:47:03.004 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-04-25T01:47:03.004 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-04-25T01:47:03.004 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-04-25T01:47:03.004 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-04-25T01:47:03.005 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-04-25T01:47:03.005 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-04-25T01:47:03.005 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-04-25T01:47:03.005 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-04-25T01:47:03.005 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-04-25T01:47:03.005 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-04-25T01:47:03.005 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-04-25T01:47:03.005 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-04-25T01:47:03.005 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-04-25T01:47:03.005 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-04-25T01:47:03.005 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-04-25T01:47:03.005 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-04-25T01:47:03.005 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-04-25T01:47:03.005 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-04-25T01:47:03.006 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-04-25T01:47:03.006 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-04-25T01:47:03.006 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-04-25T01:47:03.006 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-04-25T01:47:03.006 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-04-25T01:47:03.006 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-04-25T01:47:03.006 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-04-25T01:47:03.006 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-04-25T01:47:03.006 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-04-25T01:47:03.006 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-04-25T01:47:03.006 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-04-25T01:47:03.006 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-04-25T01:47:03.006 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-04-25T01:47:03.006 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-04-25T01:47:03.007 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-04-25T01:47:03.007 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-04-25T01:47:03.007 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-04-25T01:47:03.007 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-04-25T01:47:03.007 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-04-25T01:47:03.007 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-04-25T01:47:03.007 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-04-25T01:47:03.007 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-04-25T01:47:03.007 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-04-25T01:47:03.007 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-04-25T01:47:03.007 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-04-25T01:47:03.007 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-04-25T01:47:03.007 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-04-25T01:47:03.007 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-04-25T01:47:03.008 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-04-25T01:47:03.008 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-04-25T01:47:03.008 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-04-25T01:47:03.008 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-04-25T01:47:03.008 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-04-25T01:47:03.008 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-04-25T01:47:03.008 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-04-25T01:47:03.008 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-04-25T01:47:03.008 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-04-25T01:47:03.008 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-04-25T01:47:03.008 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-04-25T01:47:03.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-04-25T01:47:03.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-04-25T01:47:03.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-04-25T01:47:03.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-04-25T01:47:03.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-04-25T01:47:03.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-04-25T01:47:03.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-04-25T01:47:03.473 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-04-25T01:47:03.473 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-04-25T01:47:03.473 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-04-25T01:47:03.473 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-04-25T01:47:03.474 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-04-25T01:47:03.474 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-04-25T01:47:03.474 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-04-25T01:47:03.474 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-04-25T01:47:03.474 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-04-25T01:47:03.474 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-04-25T01:47:03.474 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-04-25T01:47:03.474 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-04-25T01:47:03.474 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-04-25T01:47:03.474 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-04-25T01:47:03.474 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-04-25T01:47:03.474 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-04-25T01:47:03.474 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-04-25T01:47:03.474 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-04-25T01:47:03.475 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-04-25T01:47:03.475 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-04-25T01:47:03.475 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-04-25T01:47:03.475 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-04-25T01:47:03.475 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-04-25T01:47:03.475 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-04-25T01:47:03.475 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-04-25T01:47:03.475 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-04-25T01:47:03.475 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-04-25T01:47:03.475 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-04-25T01:47:03.475 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-04-25T01:47:03.475 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-04-25T01:47:03.475 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-04-25T01:47:03.477 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-04-25T01:47:03.477 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-04-25T01:47:03.477 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-04-25T01:47:03.477 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-04-25T01:47:03.477 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-04-25T01:47:03.477 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-04-25T01:47:03.477 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-04-25T01:47:03.477 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-04-25T01:47:03.478 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-04-25T01:47:03.478 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-04-25T01:47:03.478 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-04-25T01:47:03.478 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-04-25T01:47:03.478 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-04-25T01:47:03.478 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-04-25T01:47:03.478 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-04-25T01:47:03.478 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-04-25T01:47:03.478 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-04-25T01:47:03.478 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-04-25T01:47:03.478 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-04-25T01:47:03.478 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-04-25T01:47:03.478 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-04-25T01:47:03.478 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-04-25T01:47:03.479 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-04-25T01:47:03.479 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-04-25T01:47:03.479 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-04-25T01:47:03.479 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-04-25T01:47:03.479 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-04-25T01:47:03.479 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-04-25T01:47:03.479 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-04-25T01:47:03.479 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-04-25T01:47:03.479 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-04-25T01:47:03.479 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-04-25T01:47:03.479 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-04-25T01:47:03.479 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-04-25T01:47:03.479 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-04-25T01:47:03.479 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-04-25T01:47:03.480 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-04-25T01:47:03.480 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-04-25T01:47:03.480 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-04-25T01:47:03.480 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-04-25T01:47:03.480 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-04-25T01:47:03.480 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-04-25T01:47:03.480 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-04-25T01:47:03.480 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-04-25T01:47:03.480 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-04-25T01:47:03.480 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-04-25T01:47:03.480 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-04-25T01:47:03.480 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-04-25T01:47:03.480 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-04-25T01:47:03.481 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-04-25T01:47:03.481 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-04-25T01:47:03.481 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-04-25T01:47:03.481 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-04-25T01:47:03.481 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-04-25T01:47:03.481 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-04-25T01:47:04.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:04 smithi152 ceph-mon[32465]: pgmap v407: 129 pgs: 129 active+clean; 26 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 398 KiB/s wr, 1.22k op/s 2024-04-25T01:47:04.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:04 smithi196 ceph-mon[32978]: pgmap v407: 129 pgs: 129 active+clean; 26 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 398 KiB/s wr, 1.22k op/s 2024-04-25T01:47:04.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:04 smithi078 ceph-mon[29469]: pgmap v407: 129 pgs: 129 active+clean; 26 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 398 KiB/s wr, 1.22k op/s 2024-04-25T01:47:04.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-04-25T01:47:04.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-04-25T01:47:04.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-04-25T01:47:04.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-04-25T01:47:04.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-04-25T01:47:04.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-04-25T01:47:04.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-04-25T01:47:04.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-04-25T01:47:04.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-04-25T01:47:04.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-04-25T01:47:04.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-04-25T01:47:04.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-04-25T01:47:04.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-04-25T01:47:04.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-04-25T01:47:04.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-04-25T01:47:04.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-04-25T01:47:04.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-04-25T01:47:04.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-04-25T01:47:04.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-04-25T01:47:04.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-04-25T01:47:04.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-04-25T01:47:04.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-04-25T01:47:04.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-04-25T01:47:04.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-04-25T01:47:04.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-04-25T01:47:04.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-04-25T01:47:04.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-04-25T01:47:04.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-04-25T01:47:04.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-04-25T01:47:04.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-04-25T01:47:04.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-04-25T01:47:04.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-04-25T01:47:04.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-04-25T01:47:04.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-04-25T01:47:04.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-04-25T01:47:04.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-04-25T01:47:04.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-04-25T01:47:04.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-04-25T01:47:04.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-04-25T01:47:04.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-04-25T01:47:04.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-04-25T01:47:04.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-04-25T01:47:04.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-04-25T01:47:04.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-04-25T01:47:04.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-04-25T01:47:04.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-04-25T01:47:04.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-04-25T01:47:04.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-04-25T01:47:04.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-04-25T01:47:04.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-04-25T01:47:04.528 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-04-25T01:47:04.528 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-04-25T01:47:04.528 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-04-25T01:47:04.528 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-04-25T01:47:04.528 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-04-25T01:47:04.528 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-04-25T01:47:05.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-04-25T01:47:05.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-04-25T01:47:05.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-04-25T01:47:05.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-04-25T01:47:05.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-04-25T01:47:05.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-04-25T01:47:05.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-04-25T01:47:05.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-04-25T01:47:05.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-04-25T01:47:05.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-04-25T01:47:05.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-04-25T01:47:05.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-04-25T01:47:05.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-04-25T01:47:05.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-04-25T01:47:05.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-04-25T01:47:05.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-04-25T01:47:05.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-04-25T01:47:05.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-04-25T01:47:05.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-04-25T01:47:05.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-04-25T01:47:05.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-04-25T01:47:05.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-04-25T01:47:05.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-04-25T01:47:05.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-04-25T01:47:05.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-04-25T01:47:05.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-04-25T01:47:05.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-04-25T01:47:05.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-04-25T01:47:05.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-04-25T01:47:05.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-04-25T01:47:05.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-04-25T01:47:05.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-04-25T01:47:05.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-04-25T01:47:05.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-04-25T01:47:05.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-04-25T01:47:05.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-04-25T01:47:05.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-04-25T01:47:05.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-04-25T01:47:05.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-04-25T01:47:05.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-04-25T01:47:05.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-04-25T01:47:05.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-04-25T01:47:05.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-04-25T01:47:05.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-04-25T01:47:05.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-04-25T01:47:05.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-04-25T01:47:05.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-04-25T01:47:05.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-04-25T01:47:05.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-04-25T01:47:05.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-04-25T01:47:05.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-04-25T01:47:05.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-04-25T01:47:05.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-04-25T01:47:05.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-04-25T01:47:05.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-04-25T01:47:05.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-04-25T01:47:05.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-04-25T01:47:05.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-04-25T01:47:05.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-04-25T01:47:05.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-04-25T01:47:05.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-04-25T01:47:05.828 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-04-25T01:47:05.828 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-04-25T01:47:05.828 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-04-25T01:47:05.828 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-04-25T01:47:05.828 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-04-25T01:47:05.828 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-04-25T01:47:05.828 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-04-25T01:47:05.828 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-04-25T01:47:05.828 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-04-25T01:47:05.828 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-04-25T01:47:05.828 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-04-25T01:47:05.828 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-04-25T01:47:05.828 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-04-25T01:47:05.829 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-04-25T01:47:05.829 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-04-25T01:47:05.829 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-04-25T01:47:05.829 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-04-25T01:47:05.829 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-04-25T01:47:05.829 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-04-25T01:47:05.829 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-04-25T01:47:05.829 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-04-25T01:47:06.239 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-04-25T01:47:06.239 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-04-25T01:47:06.239 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-04-25T01:47:06.239 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-04-25T01:47:06.239 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-04-25T01:47:06.239 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-04-25T01:47:06.239 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-04-25T01:47:06.239 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-04-25T01:47:06.239 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-04-25T01:47:06.239 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-04-25T01:47:06.240 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-04-25T01:47:06.240 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-04-25T01:47:06.240 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-04-25T01:47:06.240 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-04-25T01:47:06.240 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-04-25T01:47:06.240 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-04-25T01:47:06.240 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-04-25T01:47:06.240 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-04-25T01:47:06.240 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-04-25T01:47:06.240 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-04-25T01:47:06.240 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-04-25T01:47:06.240 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-04-25T01:47:06.240 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-04-25T01:47:06.240 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-04-25T01:47:06.241 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-04-25T01:47:06.241 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-04-25T01:47:06.241 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-04-25T01:47:06.241 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-04-25T01:47:06.241 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-04-25T01:47:06.241 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-04-25T01:47:06.241 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-04-25T01:47:06.241 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-04-25T01:47:06.241 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-04-25T01:47:06.241 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-04-25T01:47:06.241 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-04-25T01:47:06.241 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-04-25T01:47:06.241 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-04-25T01:47:06.241 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-04-25T01:47:06.242 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-04-25T01:47:06.242 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-04-25T01:47:06.242 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-04-25T01:47:06.242 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-04-25T01:47:06.242 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-04-25T01:47:06.242 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-04-25T01:47:06.242 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-04-25T01:47:06.242 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-04-25T01:47:06.242 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-04-25T01:47:06.242 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-04-25T01:47:06.242 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-04-25T01:47:06.242 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-04-25T01:47:06.242 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-04-25T01:47:06.243 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-04-25T01:47:06.244 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-04-25T01:47:06.244 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-04-25T01:47:06.244 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-04-25T01:47:06.244 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-04-25T01:47:06.244 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-04-25T01:47:06.244 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-04-25T01:47:06.244 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-04-25T01:47:06.244 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-04-25T01:47:06.244 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-04-25T01:47:06.244 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-04-25T01:47:06.244 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-04-25T01:47:06.244 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-04-25T01:47:06.244 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-04-25T01:47:06.244 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-04-25T01:47:06.245 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-04-25T01:47:06.245 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-04-25T01:47:06.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:06 smithi152 ceph-mon[32465]: pgmap v408: 129 pgs: 129 active+clean; 270 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 802 KiB/s wr, 1.29k op/s 2024-04-25T01:47:06.312 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:06 smithi078 ceph-mon[29469]: pgmap v408: 129 pgs: 129 active+clean; 270 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 802 KiB/s wr, 1.29k op/s 2024-04-25T01:47:06.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:06 smithi196 ceph-mon[32978]: pgmap v408: 129 pgs: 129 active+clean; 270 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 802 KiB/s wr, 1.29k op/s 2024-04-25T01:47:06.738 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-04-25T01:47:06.738 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-04-25T01:47:06.738 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-04-25T01:47:06.738 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-04-25T01:47:06.738 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-04-25T01:47:06.738 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-04-25T01:47:06.738 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-04-25T01:47:06.738 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-04-25T01:47:06.738 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-04-25T01:47:06.738 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-04-25T01:47:06.738 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-04-25T01:47:06.738 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-04-25T01:47:06.739 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-04-25T01:47:06.739 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-04-25T01:47:06.739 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-04-25T01:47:06.739 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-04-25T01:47:06.739 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-04-25T01:47:06.739 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-04-25T01:47:06.739 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-04-25T01:47:06.739 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-04-25T01:47:06.739 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-04-25T01:47:06.739 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-04-25T01:47:06.739 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-04-25T01:47:06.739 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-04-25T01:47:06.739 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-04-25T01:47:06.740 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-04-25T01:47:06.741 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-04-25T01:47:06.741 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-04-25T01:47:06.741 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-04-25T01:47:06.741 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-04-25T01:47:06.741 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-04-25T01:47:06.741 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-04-25T01:47:06.741 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-04-25T01:47:06.741 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-04-25T01:47:06.741 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-04-25T01:47:06.741 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-04-25T01:47:06.741 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-04-25T01:47:06.741 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-04-25T01:47:06.741 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-04-25T01:47:06.741 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-04-25T01:47:06.742 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-04-25T01:47:06.742 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-04-25T01:47:06.742 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-04-25T01:47:06.742 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-04-25T01:47:06.742 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-04-25T01:47:06.742 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-04-25T01:47:06.742 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-04-25T01:47:06.742 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-04-25T01:47:06.742 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-04-25T01:47:06.742 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-04-25T01:47:06.742 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-04-25T01:47:06.742 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-04-25T01:47:06.742 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-04-25T01:47:06.742 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-04-25T01:47:06.743 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-04-25T01:47:07.906 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-04-25T01:47:07.906 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-04-25T01:47:07.906 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-04-25T01:47:07.906 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-04-25T01:47:07.906 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-04-25T01:47:07.906 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-04-25T01:47:07.907 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-04-25T01:47:07.907 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-04-25T01:47:07.907 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-04-25T01:47:07.907 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-04-25T01:47:07.907 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-04-25T01:47:07.907 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-04-25T01:47:07.907 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-04-25T01:47:07.907 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-04-25T01:47:07.907 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-04-25T01:47:07.907 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-04-25T01:47:07.907 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-04-25T01:47:07.907 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-04-25T01:47:07.907 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-04-25T01:47:07.907 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-04-25T01:47:07.908 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-04-25T01:47:07.909 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-04-25T01:47:07.910 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-04-25T01:47:07.910 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-04-25T01:47:07.910 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-04-25T01:47:07.910 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-04-25T01:47:07.910 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-04-25T01:47:07.910 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-04-25T01:47:07.910 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-04-25T01:47:07.910 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-04-25T01:47:07.910 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-04-25T01:47:07.910 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-04-25T01:47:07.910 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-04-25T01:47:07.910 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-04-25T01:47:07.910 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-04-25T01:47:07.910 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-04-25T01:47:07.911 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-04-25T01:47:07.911 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-04-25T01:47:07.911 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-04-25T01:47:07.911 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-04-25T01:47:07.911 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-04-25T01:47:07.911 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-04-25T01:47:07.911 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-04-25T01:47:07.911 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-04-25T01:47:07.911 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-04-25T01:47:07.911 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-04-25T01:47:07.911 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-04-25T01:47:07.911 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-04-25T01:47:07.911 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-04-25T01:47:07.911 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-04-25T01:47:07.912 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-04-25T01:47:07.912 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-04-25T01:47:08.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:08 smithi152 ceph-mon[32465]: pgmap v409: 129 pgs: 129 active+clean; 278 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 832 KiB/s wr, 689 op/s 2024-04-25T01:47:08.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:08 smithi196 ceph-mon[32978]: pgmap v409: 129 pgs: 129 active+clean; 278 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 832 KiB/s wr, 689 op/s 2024-04-25T01:47:08.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:08 smithi078 ceph-mon[29469]: pgmap v409: 129 pgs: 129 active+clean; 278 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 832 KiB/s wr, 689 op/s 2024-04-25T01:47:08.821 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-04-25T01:47:08.822 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-04-25T01:47:08.822 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-04-25T01:47:08.822 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-04-25T01:47:08.822 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-04-25T01:47:08.822 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-04-25T01:47:08.822 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-04-25T01:47:08.822 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-04-25T01:47:08.822 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-04-25T01:47:08.822 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-04-25T01:47:08.822 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-04-25T01:47:08.822 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-04-25T01:47:08.822 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-04-25T01:47:08.822 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-04-25T01:47:08.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-04-25T01:47:08.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-04-25T01:47:08.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-04-25T01:47:08.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-04-25T01:47:08.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-04-25T01:47:08.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-04-25T01:47:08.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-04-25T01:47:08.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-04-25T01:47:08.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-04-25T01:47:08.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-04-25T01:47:08.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-04-25T01:47:08.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-04-25T01:47:08.823 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-04-25T01:47:08.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-04-25T01:47:08.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-04-25T01:47:08.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-04-25T01:47:08.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-04-25T01:47:08.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-04-25T01:47:08.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-04-25T01:47:08.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-04-25T01:47:08.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-04-25T01:47:08.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-04-25T01:47:08.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-04-25T01:47:08.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-04-25T01:47:08.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-04-25T01:47:08.824 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-04-25T01:47:08.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-04-25T01:47:08.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-04-25T01:47:08.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-04-25T01:47:08.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-04-25T01:47:08.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-04-25T01:47:08.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-04-25T01:47:08.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-04-25T01:47:08.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-04-25T01:47:08.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-04-25T01:47:08.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-04-25T01:47:08.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-04-25T01:47:08.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-04-25T01:47:08.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-04-25T01:47:08.825 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-04-25T01:47:08.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-04-25T01:47:08.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-04-25T01:47:08.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-04-25T01:47:08.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-04-25T01:47:08.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-04-25T01:47:08.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-04-25T01:47:08.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-04-25T01:47:08.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-04-25T01:47:08.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-04-25T01:47:08.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-04-25T01:47:08.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-04-25T01:47:08.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-04-25T01:47:08.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-04-25T01:47:08.826 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-04-25T01:47:08.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-04-25T01:47:08.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-04-25T01:47:08.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-04-25T01:47:08.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-04-25T01:47:08.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-04-25T01:47:08.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-04-25T01:47:08.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-04-25T01:47:08.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-04-25T01:47:08.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-04-25T01:47:08.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-04-25T01:47:08.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-04-25T01:47:08.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-04-25T01:47:08.827 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-04-25T01:47:09.304 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-04-25T01:47:09.304 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-04-25T01:47:09.304 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-04-25T01:47:09.304 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-04-25T01:47:09.304 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-04-25T01:47:09.304 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-04-25T01:47:09.304 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-04-25T01:47:09.304 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-04-25T01:47:09.305 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-04-25T01:47:09.306 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-04-25T01:47:09.306 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-04-25T01:47:09.306 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-04-25T01:47:09.306 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-04-25T01:47:09.306 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-04-25T01:47:09.306 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-04-25T01:47:09.306 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-04-25T01:47:09.306 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-04-25T01:47:09.306 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-04-25T01:47:09.306 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-04-25T01:47:09.306 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-04-25T01:47:09.306 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-04-25T01:47:09.306 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-04-25T01:47:09.307 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-04-25T01:47:09.308 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-04-25T01:47:09.308 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-04-25T01:47:09.308 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-04-25T01:47:09.308 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-04-25T01:47:09.308 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-04-25T01:47:09.308 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-04-25T01:47:09.308 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-04-25T01:47:09.308 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-04-25T01:47:09.308 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-04-25T01:47:09.308 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-04-25T01:47:09.308 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-04-25T01:47:09.308 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-04-25T01:47:09.308 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-04-25T01:47:09.308 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-04-25T01:47:09.309 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-04-25T01:47:09.309 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-04-25T01:47:09.309 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-04-25T01:47:09.309 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-04-25T01:47:09.309 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-04-25T01:47:09.309 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-04-25T01:47:09.309 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-04-25T01:47:09.309 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-04-25T01:47:09.310 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-04-25T01:47:09.310 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-04-25T01:47:09.310 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-04-25T01:47:09.310 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-04-25T01:47:09.310 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-04-25T01:47:09.310 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-04-25T01:47:09.310 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-04-25T01:47:09.310 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-04-25T01:47:10.048 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-04-25T01:47:10.048 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-04-25T01:47:10.048 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-04-25T01:47:10.048 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-04-25T01:47:10.048 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-04-25T01:47:10.048 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-04-25T01:47:10.048 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-04-25T01:47:10.048 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-04-25T01:47:10.048 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-04-25T01:47:10.049 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-04-25T01:47:10.049 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-04-25T01:47:10.049 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-04-25T01:47:10.049 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-04-25T01:47:10.049 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-04-25T01:47:10.049 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-04-25T01:47:10.049 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-04-25T01:47:10.049 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-04-25T01:47:10.049 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-04-25T01:47:10.049 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-04-25T01:47:10.049 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-04-25T01:47:10.049 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-04-25T01:47:10.049 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-04-25T01:47:10.049 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-04-25T01:47:10.050 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-04-25T01:47:10.050 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-04-25T01:47:10.050 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-04-25T01:47:10.050 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-04-25T01:47:10.050 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-04-25T01:47:10.050 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-04-25T01:47:10.050 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-04-25T01:47:10.050 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-04-25T01:47:10.050 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-04-25T01:47:10.050 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-04-25T01:47:10.050 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-04-25T01:47:10.050 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-04-25T01:47:10.050 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-04-25T01:47:10.050 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-04-25T01:47:10.051 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-04-25T01:47:10.051 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-04-25T01:47:10.051 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-04-25T01:47:10.051 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-04-25T01:47:10.051 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-04-25T01:47:10.051 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-04-25T01:47:10.051 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-04-25T01:47:10.051 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-04-25T01:47:10.051 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-04-25T01:47:10.051 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-04-25T01:47:10.051 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-04-25T01:47:10.051 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-04-25T01:47:10.051 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-04-25T01:47:10.051 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-04-25T01:47:10.052 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-04-25T01:47:10.052 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-04-25T01:47:10.052 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-04-25T01:47:10.052 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-04-25T01:47:10.052 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-04-25T01:47:10.052 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-04-25T01:47:10.052 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-04-25T01:47:10.052 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-04-25T01:47:10.052 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-04-25T01:47:10.052 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-04-25T01:47:10.052 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-04-25T01:47:10.052 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-04-25T01:47:10.052 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-04-25T01:47:10.052 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-04-25T01:47:10.053 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-04-25T01:47:10.054 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-04-25T01:47:10.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:10 smithi152 ceph-mon[32465]: pgmap v410: 129 pgs: 129 active+clean; 305 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 1.1 MiB/s wr, 749 op/s 2024-04-25T01:47:10.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:10 smithi196 ceph-mon[32978]: pgmap v410: 129 pgs: 129 active+clean; 305 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 1.1 MiB/s wr, 749 op/s 2024-04-25T01:47:10.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:10 smithi078 ceph-mon[29469]: pgmap v410: 129 pgs: 129 active+clean; 305 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 1.1 MiB/s wr, 749 op/s 2024-04-25T01:47:11.262 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-04-25T01:47:11.263 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-04-25T01:47:11.263 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-04-25T01:47:11.263 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-04-25T01:47:11.263 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-04-25T01:47:11.263 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-04-25T01:47:11.263 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-04-25T01:47:11.263 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-04-25T01:47:11.263 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-04-25T01:47:11.263 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-04-25T01:47:11.263 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-04-25T01:47:11.263 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-04-25T01:47:11.263 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-04-25T01:47:11.264 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-04-25T01:47:11.264 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-04-25T01:47:11.264 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-04-25T01:47:11.264 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-04-25T01:47:11.264 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-04-25T01:47:11.264 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-04-25T01:47:11.264 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-04-25T01:47:11.264 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-04-25T01:47:11.264 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-04-25T01:47:11.264 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-04-25T01:47:11.264 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-04-25T01:47:11.264 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-04-25T01:47:11.264 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-04-25T01:47:11.264 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-04-25T01:47:11.265 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-04-25T01:47:11.265 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-04-25T01:47:11.265 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-04-25T01:47:11.265 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-04-25T01:47:11.265 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-04-25T01:47:11.265 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-04-25T01:47:11.265 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-04-25T01:47:11.265 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-04-25T01:47:11.265 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-04-25T01:47:11.265 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-04-25T01:47:11.265 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-04-25T01:47:11.265 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-04-25T01:47:11.265 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-04-25T01:47:11.265 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-04-25T01:47:11.266 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-04-25T01:47:11.266 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-04-25T01:47:11.266 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-04-25T01:47:11.266 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-04-25T01:47:11.266 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-04-25T01:47:11.266 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-04-25T01:47:11.266 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-04-25T01:47:11.266 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-04-25T01:47:11.266 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-04-25T01:47:11.266 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-04-25T01:47:11.266 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-04-25T01:47:11.266 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-04-25T01:47:11.266 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-04-25T01:47:11.266 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-04-25T01:47:11.267 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-04-25T01:47:11.267 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-04-25T01:47:11.267 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-04-25T01:47:11.267 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-04-25T01:47:11.267 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-04-25T01:47:11.267 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-04-25T01:47:11.267 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-04-25T01:47:11.267 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-04-25T01:47:11.267 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-04-25T01:47:11.267 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-04-25T01:47:11.267 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-04-25T01:47:11.267 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-04-25T01:47:11.267 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-04-25T01:47:11.267 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-04-25T01:47:11.268 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-04-25T01:47:11.268 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-04-25T01:47:11.268 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-04-25T01:47:11.268 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-04-25T01:47:11.268 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-04-25T01:47:11.268 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-04-25T01:47:11.268 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-04-25T01:47:11.268 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-04-25T01:47:11.268 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-04-25T01:47:11.268 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-04-25T01:47:11.268 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-04-25T01:47:11.268 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-04-25T01:47:12.008 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-04-25T01:47:12.008 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-04-25T01:47:12.008 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-04-25T01:47:12.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-04-25T01:47:12.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-04-25T01:47:12.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-04-25T01:47:12.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-04-25T01:47:12.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-04-25T01:47:12.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-04-25T01:47:12.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-04-25T01:47:12.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-04-25T01:47:12.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-04-25T01:47:12.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-04-25T01:47:12.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-04-25T01:47:12.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-04-25T01:47:12.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-04-25T01:47:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-04-25T01:47:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-04-25T01:47:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-04-25T01:47:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-04-25T01:47:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-04-25T01:47:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-04-25T01:47:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-04-25T01:47:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-04-25T01:47:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-04-25T01:47:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-04-25T01:47:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-04-25T01:47:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-04-25T01:47:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-04-25T01:47:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-04-25T01:47:12.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-04-25T01:47:12.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-04-25T01:47:12.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-04-25T01:47:12.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-04-25T01:47:12.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-04-25T01:47:12.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-04-25T01:47:12.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-04-25T01:47:12.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-04-25T01:47:12.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-04-25T01:47:12.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-04-25T01:47:12.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-04-25T01:47:12.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-04-25T01:47:12.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-04-25T01:47:12.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-04-25T01:47:12.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-04-25T01:47:12.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-04-25T01:47:12.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-04-25T01:47:12.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-04-25T01:47:12.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-04-25T01:47:12.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-04-25T01:47:12.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-04-25T01:47:12.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-04-25T01:47:12.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-04-25T01:47:12.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-04-25T01:47:12.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-04-25T01:47:12.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-04-25T01:47:12.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-04-25T01:47:12.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-04-25T01:47:12.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-04-25T01:47:12.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-04-25T01:47:12.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-04-25T01:47:12.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-04-25T01:47:12.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-04-25T01:47:12.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-04-25T01:47:12.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-04-25T01:47:12.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-04-25T01:47:12.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-04-25T01:47:12.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-04-25T01:47:12.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-04-25T01:47:12.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-04-25T01:47:12.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-04-25T01:47:12.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-04-25T01:47:12.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-04-25T01:47:12.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-04-25T01:47:12.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-04-25T01:47:12.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-04-25T01:47:12.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-04-25T01:47:12.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-04-25T01:47:12.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-04-25T01:47:12.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-04-25T01:47:12.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:12 smithi152 ceph-mon[32465]: pgmap v411: 129 pgs: 129 active+clean; 277 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s rd, 1.2 MiB/s wr, 801 op/s 2024-04-25T01:47:12.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:12 smithi196 ceph-mon[32978]: pgmap v411: 129 pgs: 129 active+clean; 277 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s rd, 1.2 MiB/s wr, 801 op/s 2024-04-25T01:47:12.408 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-04-25T01:47:12.408 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-04-25T01:47:12.408 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-04-25T01:47:12.408 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-04-25T01:47:12.408 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-04-25T01:47:12.409 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-04-25T01:47:12.409 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-04-25T01:47:12.409 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-04-25T01:47:12.409 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-04-25T01:47:12.409 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-04-25T01:47:12.409 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-04-25T01:47:12.409 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-04-25T01:47:12.409 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-04-25T01:47:12.409 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-04-25T01:47:12.409 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-04-25T01:47:12.409 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-04-25T01:47:12.409 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-04-25T01:47:12.409 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-04-25T01:47:12.409 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-04-25T01:47:12.410 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-04-25T01:47:12.410 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-04-25T01:47:12.410 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-04-25T01:47:12.410 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-04-25T01:47:12.410 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-04-25T01:47:12.410 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-04-25T01:47:12.410 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-04-25T01:47:12.410 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-04-25T01:47:12.410 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-04-25T01:47:12.410 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-04-25T01:47:12.410 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-04-25T01:47:12.410 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-04-25T01:47:12.410 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-04-25T01:47:12.410 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-04-25T01:47:12.411 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-04-25T01:47:12.411 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-04-25T01:47:12.411 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-04-25T01:47:12.411 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-04-25T01:47:12.411 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-04-25T01:47:12.411 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-04-25T01:47:12.411 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-04-25T01:47:12.411 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-04-25T01:47:12.411 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-04-25T01:47:12.411 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-04-25T01:47:12.411 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-04-25T01:47:12.411 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-04-25T01:47:12.411 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-04-25T01:47:12.412 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-04-25T01:47:12.412 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-04-25T01:47:12.412 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-04-25T01:47:12.412 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-04-25T01:47:12.412 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-04-25T01:47:12.412 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-04-25T01:47:12.412 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-04-25T01:47:12.412 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-04-25T01:47:12.412 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-04-25T01:47:12.412 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-04-25T01:47:12.412 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-04-25T01:47:12.412 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-04-25T01:47:12.412 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-04-25T01:47:12.412 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-04-25T01:47:12.413 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-04-25T01:47:12.414 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-04-25T01:47:12.414 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-04-25T01:47:12.414 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-04-25T01:47:12.414 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-04-25T01:47:12.414 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-04-25T01:47:12.414 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-04-25T01:47:12.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:12 smithi078 ceph-mon[29469]: pgmap v411: 129 pgs: 129 active+clean; 277 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 68 MiB/s rd, 1.2 MiB/s wr, 801 op/s 2024-04-25T01:47:13.326 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-04-25T01:47:13.326 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-04-25T01:47:13.326 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-04-25T01:47:13.326 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-04-25T01:47:13.326 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-04-25T01:47:13.326 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-04-25T01:47:13.326 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-04-25T01:47:13.327 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-04-25T01:47:13.328 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-04-25T01:47:13.328 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-04-25T01:47:13.328 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-04-25T01:47:13.328 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-04-25T01:47:13.328 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-04-25T01:47:13.328 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-04-25T01:47:13.328 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-04-25T01:47:13.328 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-04-25T01:47:13.328 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-04-25T01:47:13.328 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-04-25T01:47:13.328 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-04-25T01:47:13.328 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-04-25T01:47:13.328 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-04-25T01:47:13.328 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-04-25T01:47:13.329 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-04-25T01:47:13.329 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-04-25T01:47:13.329 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-04-25T01:47:13.329 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-04-25T01:47:13.329 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-04-25T01:47:13.329 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-04-25T01:47:13.329 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-04-25T01:47:13.329 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-04-25T01:47:13.329 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-04-25T01:47:13.329 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-04-25T01:47:13.329 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-04-25T01:47:13.329 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-04-25T01:47:13.329 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-04-25T01:47:13.329 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-04-25T01:47:13.330 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-04-25T01:47:13.330 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-04-25T01:47:13.330 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-04-25T01:47:13.330 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-04-25T01:47:13.330 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-04-25T01:47:13.330 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-04-25T01:47:13.330 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-04-25T01:47:13.330 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-04-25T01:47:13.330 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-04-25T01:47:13.330 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-04-25T01:47:13.330 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-04-25T01:47:13.330 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-04-25T01:47:13.330 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-04-25T01:47:13.330 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-04-25T01:47:13.331 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-04-25T01:47:13.332 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-04-25T01:47:13.332 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-04-25T01:47:14.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:14 smithi152 ceph-mon[32465]: pgmap v412: 129 pgs: 129 active+clean; 282 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s rd, 1.3 MiB/s wr, 352 op/s 2024-04-25T01:47:14.369 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:14 smithi196 ceph-mon[32978]: pgmap v412: 129 pgs: 129 active+clean; 282 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s rd, 1.3 MiB/s wr, 352 op/s 2024-04-25T01:47:14.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:14 smithi078 ceph-mon[29469]: pgmap v412: 129 pgs: 129 active+clean; 282 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 72 MiB/s rd, 1.3 MiB/s wr, 352 op/s 2024-04-25T01:47:14.576 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-04-25T01:47:14.576 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-04-25T01:47:14.576 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-04-25T01:47:14.576 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-04-25T01:47:14.576 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-04-25T01:47:14.576 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-04-25T01:47:14.576 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-04-25T01:47:14.576 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-04-25T01:47:14.576 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-04-25T01:47:14.576 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-04-25T01:47:14.576 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-04-25T01:47:14.577 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-04-25T01:47:14.578 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-04-25T01:47:14.578 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-04-25T01:47:14.578 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-04-25T01:47:14.578 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-04-25T01:47:14.578 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-04-25T01:47:14.578 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-04-25T01:47:14.578 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-04-25T01:47:14.578 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-04-25T01:47:14.578 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-04-25T01:47:14.578 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-04-25T01:47:14.578 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-04-25T01:47:14.578 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-04-25T01:47:14.578 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-04-25T01:47:14.578 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-04-25T01:47:14.579 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-04-25T01:47:14.579 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-04-25T01:47:14.579 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-04-25T01:47:14.579 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-04-25T01:47:14.579 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-04-25T01:47:14.579 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-04-25T01:47:14.579 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-04-25T01:47:14.579 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-04-25T01:47:14.579 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-04-25T01:47:14.579 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-04-25T01:47:14.579 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-04-25T01:47:14.579 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-04-25T01:47:14.579 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-04-25T01:47:14.579 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-04-25T01:47:14.580 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-04-25T01:47:14.581 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-04-25T01:47:14.581 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-04-25T01:47:14.581 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-04-25T01:47:14.581 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-04-25T01:47:14.581 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-04-25T01:47:14.581 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-04-25T01:47:14.581 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-04-25T01:47:14.581 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-04-25T01:47:14.581 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-04-25T01:47:14.581 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-04-25T01:47:14.581 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-04-25T01:47:14.581 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-04-25T01:47:15.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-04-25T01:47:15.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-04-25T01:47:15.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-04-25T01:47:15.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-04-25T01:47:15.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-04-25T01:47:15.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-04-25T01:47:15.009 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-04-25T01:47:15.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-04-25T01:47:15.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-04-25T01:47:15.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-04-25T01:47:15.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-04-25T01:47:15.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-04-25T01:47:15.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-04-25T01:47:15.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-04-25T01:47:15.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-04-25T01:47:15.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-04-25T01:47:15.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-04-25T01:47:15.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-04-25T01:47:15.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-04-25T01:47:15.010 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-04-25T01:47:15.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-04-25T01:47:15.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-04-25T01:47:15.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-04-25T01:47:15.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-04-25T01:47:15.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-04-25T01:47:15.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-04-25T01:47:15.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-04-25T01:47:15.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-04-25T01:47:15.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-04-25T01:47:15.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-04-25T01:47:15.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-04-25T01:47:15.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-04-25T01:47:15.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-04-25T01:47:15.011 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-04-25T01:47:15.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-04-25T01:47:15.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-04-25T01:47:15.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-04-25T01:47:15.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-04-25T01:47:15.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-04-25T01:47:15.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-04-25T01:47:15.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-04-25T01:47:15.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-04-25T01:47:15.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-04-25T01:47:15.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-04-25T01:47:15.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-04-25T01:47:15.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-04-25T01:47:15.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-04-25T01:47:15.012 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-04-25T01:47:15.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-04-25T01:47:15.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-04-25T01:47:15.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-04-25T01:47:15.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-04-25T01:47:15.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-04-25T01:47:15.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-04-25T01:47:15.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-04-25T01:47:15.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-04-25T01:47:15.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-04-25T01:47:15.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-04-25T01:47:15.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-04-25T01:47:15.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-04-25T01:47:15.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-04-25T01:47:15.013 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-04-25T01:47:15.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-04-25T01:47:15.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-04-25T01:47:15.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-04-25T01:47:15.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-04-25T01:47:15.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-04-25T01:47:15.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-04-25T01:47:15.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-04-25T01:47:15.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-04-25T01:47:15.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-04-25T01:47:15.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-04-25T01:47:15.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-04-25T01:47:15.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-04-25T01:47:15.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-04-25T01:47:15.014 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-04-25T01:47:15.015 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-04-25T01:47:15.015 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-04-25T01:47:15.015 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-04-25T01:47:15.015 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-04-25T01:47:15.521 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-04-25T01:47:15.521 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-04-25T01:47:15.521 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-04-25T01:47:15.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-04-25T01:47:15.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-04-25T01:47:15.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-04-25T01:47:15.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-04-25T01:47:15.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-04-25T01:47:15.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-04-25T01:47:15.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-04-25T01:47:15.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-04-25T01:47:15.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-04-25T01:47:15.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-04-25T01:47:15.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-04-25T01:47:15.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-04-25T01:47:15.522 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-04-25T01:47:15.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-04-25T01:47:15.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-04-25T01:47:15.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-04-25T01:47:15.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-04-25T01:47:15.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-04-25T01:47:15.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-04-25T01:47:15.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-04-25T01:47:15.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-04-25T01:47:15.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-04-25T01:47:15.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-04-25T01:47:15.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-04-25T01:47:15.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-04-25T01:47:15.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-04-25T01:47:15.523 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-04-25T01:47:15.524 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-04-25T01:47:15.525 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-04-25T01:47:15.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-04-25T01:47:15.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-04-25T01:47:15.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-04-25T01:47:15.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-04-25T01:47:15.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-04-25T01:47:15.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-04-25T01:47:15.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-04-25T01:47:15.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-04-25T01:47:15.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-04-25T01:47:15.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-04-25T01:47:15.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-04-25T01:47:15.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-04-25T01:47:15.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-04-25T01:47:15.526 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-04-25T01:47:15.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-04-25T01:47:15.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-04-25T01:47:15.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-04-25T01:47:15.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-04-25T01:47:15.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-04-25T01:47:15.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-04-25T01:47:15.527 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-04-25T01:47:16.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:16 smithi152 ceph-mon[32465]: pgmap v413: 129 pgs: 129 active+clean; 300 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s rd, 1.2 MiB/s wr, 359 op/s 2024-04-25T01:47:16.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:16 smithi196 ceph-mon[32978]: pgmap v413: 129 pgs: 129 active+clean; 300 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s rd, 1.2 MiB/s wr, 359 op/s 2024-04-25T01:47:16.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:16 smithi078 ceph-mon[29469]: pgmap v413: 129 pgs: 129 active+clean; 300 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s rd, 1.2 MiB/s wr, 359 op/s 2024-04-25T01:47:16.676 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-04-25T01:47:16.676 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-04-25T01:47:16.676 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-04-25T01:47:16.676 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-04-25T01:47:16.676 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-04-25T01:47:16.677 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-04-25T01:47:16.677 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-04-25T01:47:16.677 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-04-25T01:47:16.677 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-04-25T01:47:16.677 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-04-25T01:47:16.677 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-04-25T01:47:16.677 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-04-25T01:47:16.677 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-04-25T01:47:16.677 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-04-25T01:47:16.677 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-04-25T01:47:16.677 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-04-25T01:47:16.677 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-04-25T01:47:16.677 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-04-25T01:47:16.678 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-04-25T01:47:16.678 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-04-25T01:47:16.678 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-04-25T01:47:16.678 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-04-25T01:47:16.678 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-04-25T01:47:16.678 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-04-25T01:47:16.678 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-04-25T01:47:16.678 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-04-25T01:47:16.678 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-04-25T01:47:16.678 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-04-25T01:47:16.678 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-04-25T01:47:16.678 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-04-25T01:47:16.678 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-04-25T01:47:16.679 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-04-25T01:47:16.679 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-04-25T01:47:16.679 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-04-25T01:47:16.679 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-04-25T01:47:16.679 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-04-25T01:47:16.679 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-04-25T01:47:16.679 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-04-25T01:47:16.679 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-04-25T01:47:16.679 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-04-25T01:47:16.679 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-04-25T01:47:16.679 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-04-25T01:47:16.679 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-04-25T01:47:16.679 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-04-25T01:47:16.680 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-04-25T01:47:16.680 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-04-25T01:47:16.680 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-04-25T01:47:16.680 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-04-25T01:47:16.680 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-04-25T01:47:16.680 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-04-25T01:47:16.680 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-04-25T01:47:16.680 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-04-25T01:47:16.680 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-04-25T01:47:16.680 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-04-25T01:47:16.680 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-04-25T01:47:16.680 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-04-25T01:47:16.680 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-04-25T01:47:16.680 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-04-25T01:47:16.681 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-04-25T01:47:16.681 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-04-25T01:47:16.681 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-04-25T01:47:16.681 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-04-25T01:47:16.681 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-04-25T01:47:16.681 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-04-25T01:47:16.681 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-04-25T01:47:16.681 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-04-25T01:47:16.681 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-04-25T01:47:16.681 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-04-25T01:47:16.681 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-04-25T01:47:16.681 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-04-25T01:47:16.681 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-04-25T01:47:16.681 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-04-25T01:47:16.682 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-04-25T01:47:16.682 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-04-25T01:47:16.682 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-04-25T01:47:16.682 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-04-25T01:47:16.682 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-04-25T01:47:16.682 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-04-25T01:47:16.682 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-04-25T01:47:16.682 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-04-25T01:47:16.682 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-04-25T01:47:17.666 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-04-25T01:47:17.667 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-04-25T01:47:17.667 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-04-25T01:47:17.667 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-04-25T01:47:17.667 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-04-25T01:47:17.667 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-04-25T01:47:17.667 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-04-25T01:47:17.667 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-04-25T01:47:17.667 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-04-25T01:47:17.667 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-04-25T01:47:17.667 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-04-25T01:47:17.667 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-04-25T01:47:17.667 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-04-25T01:47:17.668 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-04-25T01:47:17.668 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-04-25T01:47:17.668 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-04-25T01:47:17.668 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-04-25T01:47:17.668 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-04-25T01:47:17.668 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-04-25T01:47:17.668 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-04-25T01:47:17.668 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-04-25T01:47:17.668 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-04-25T01:47:17.668 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-04-25T01:47:17.668 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-04-25T01:47:17.668 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-04-25T01:47:17.668 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-04-25T01:47:17.668 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-04-25T01:47:17.669 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-04-25T01:47:17.670 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-04-25T01:47:17.670 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-04-25T01:47:17.670 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-04-25T01:47:17.670 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-04-25T01:47:17.670 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-04-25T01:47:17.670 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-04-25T01:47:17.670 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-04-25T01:47:17.670 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-04-25T01:47:17.670 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-04-25T01:47:17.670 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-04-25T01:47:17.670 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-04-25T01:47:17.671 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-04-25T01:47:17.671 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-04-25T01:47:17.671 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-04-25T01:47:17.671 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-04-25T01:47:17.671 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-04-25T01:47:17.671 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-04-25T01:47:17.671 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-04-25T01:47:17.671 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-04-25T01:47:17.671 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-04-25T01:47:17.671 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-04-25T01:47:17.671 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-04-25T01:47:17.672 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-04-25T01:47:17.673 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-04-25T01:47:17.673 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-04-25T01:47:18.075 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-04-25T01:47:18.075 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-04-25T01:47:18.075 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-04-25T01:47:18.075 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-04-25T01:47:18.075 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-04-25T01:47:18.075 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-04-25T01:47:18.075 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-04-25T01:47:18.075 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-04-25T01:47:18.075 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-04-25T01:47:18.075 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-04-25T01:47:18.075 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-04-25T01:47:18.076 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-04-25T01:47:18.077 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-04-25T01:47:18.077 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-04-25T01:47:18.077 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-04-25T01:47:18.077 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-04-25T01:47:18.077 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-04-25T01:47:18.077 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-04-25T01:47:18.077 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-04-25T01:47:18.077 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-04-25T01:47:18.077 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-04-25T01:47:18.077 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-04-25T01:47:18.077 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-04-25T01:47:18.077 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-04-25T01:47:18.077 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-04-25T01:47:18.077 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-04-25T01:47:18.078 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-04-25T01:47:18.078 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-04-25T01:47:18.078 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-04-25T01:47:18.078 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-04-25T01:47:18.078 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-04-25T01:47:18.078 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-04-25T01:47:18.078 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-04-25T01:47:18.078 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-04-25T01:47:18.079 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-04-25T01:47:18.079 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-04-25T01:47:18.079 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-04-25T01:47:18.079 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-04-25T01:47:18.079 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-04-25T01:47:18.079 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-04-25T01:47:18.079 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-04-25T01:47:18.079 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-04-25T01:47:18.079 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-04-25T01:47:18.079 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-04-25T01:47:18.079 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-04-25T01:47:18.079 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-04-25T01:47:18.080 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-04-25T01:47:18.080 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-04-25T01:47:18.080 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-04-25T01:47:18.080 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-04-25T01:47:18.080 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-04-25T01:47:18.080 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-04-25T01:47:18.080 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-04-25T01:47:18.080 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-04-25T01:47:18.080 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-04-25T01:47:18.080 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-04-25T01:47:18.080 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-04-25T01:47:18.080 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-04-25T01:47:18.080 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-04-25T01:47:18.081 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-04-25T01:47:18.081 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-04-25T01:47:18.081 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-04-25T01:47:18.081 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-04-25T01:47:18.081 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-04-25T01:47:18.081 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-04-25T01:47:18.081 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-04-25T01:47:18.081 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-04-25T01:47:18.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:18 smithi152 ceph-mon[32465]: pgmap v414: 129 pgs: 129 active+clean; 340 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 96 MiB/s rd, 884 KiB/s wr, 300 op/s 2024-04-25T01:47:18.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:18 smithi196 ceph-mon[32978]: pgmap v414: 129 pgs: 129 active+clean; 340 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 96 MiB/s rd, 884 KiB/s wr, 300 op/s 2024-04-25T01:47:18.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:18 smithi078 ceph-mon[29469]: pgmap v414: 129 pgs: 129 active+clean; 340 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 96 MiB/s rd, 884 KiB/s wr, 300 op/s 2024-04-25T01:47:18.653 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-04-25T01:47:18.653 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-04-25T01:47:18.653 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-04-25T01:47:18.653 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-04-25T01:47:18.653 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-04-25T01:47:18.653 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-04-25T01:47:18.653 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-04-25T01:47:18.653 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-04-25T01:47:18.653 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-04-25T01:47:18.654 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-04-25T01:47:18.654 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-04-25T01:47:18.654 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-04-25T01:47:18.654 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-04-25T01:47:18.654 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-04-25T01:47:18.654 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-04-25T01:47:18.654 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-04-25T01:47:18.654 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-04-25T01:47:18.654 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-04-25T01:47:18.654 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-04-25T01:47:18.654 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-04-25T01:47:18.654 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-04-25T01:47:18.655 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-04-25T01:47:18.655 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-04-25T01:47:18.655 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-04-25T01:47:18.655 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-04-25T01:47:18.655 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-04-25T01:47:18.655 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-04-25T01:47:18.655 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-04-25T01:47:18.655 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-04-25T01:47:18.655 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-04-25T01:47:18.655 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-04-25T01:47:18.655 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-04-25T01:47:18.655 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-04-25T01:47:18.655 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-04-25T01:47:18.655 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-04-25T01:47:18.656 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-04-25T01:47:18.656 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-04-25T01:47:18.656 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-04-25T01:47:18.656 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-04-25T01:47:18.656 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-04-25T01:47:18.656 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-04-25T01:47:18.656 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-04-25T01:47:18.656 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-04-25T01:47:18.656 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-04-25T01:47:18.656 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-04-25T01:47:18.656 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-04-25T01:47:18.656 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-04-25T01:47:18.657 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-04-25T01:47:18.657 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-04-25T01:47:18.657 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-04-25T01:47:18.657 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-04-25T01:47:18.657 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-04-25T01:47:18.657 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-04-25T01:47:18.657 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-04-25T01:47:18.657 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-04-25T01:47:18.657 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-04-25T01:47:18.657 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-04-25T01:47:18.657 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-04-25T01:47:18.658 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-04-25T01:47:18.658 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-04-25T01:47:18.658 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-04-25T01:47:18.658 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-04-25T01:47:18.658 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-04-25T01:47:18.658 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-04-25T01:47:18.658 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-04-25T01:47:18.658 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-04-25T01:47:18.658 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-04-25T01:47:18.658 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-04-25T01:47:18.658 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-04-25T01:47:18.658 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-04-25T01:47:18.658 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-04-25T01:47:18.659 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-04-25T01:47:18.659 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-04-25T01:47:18.659 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-04-25T01:47:18.659 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-04-25T01:47:18.659 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-04-25T01:47:18.659 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-04-25T01:47:18.659 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-04-25T01:47:18.659 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-04-25T01:47:18.659 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-04-25T01:47:19.885 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-04-25T01:47:19.886 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-04-25T01:47:19.886 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-04-25T01:47:19.886 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-04-25T01:47:19.886 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-04-25T01:47:19.886 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-04-25T01:47:19.886 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-04-25T01:47:19.886 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-04-25T01:47:19.886 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-04-25T01:47:19.886 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-04-25T01:47:19.886 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-04-25T01:47:19.886 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-04-25T01:47:19.886 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-04-25T01:47:19.886 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-04-25T01:47:19.887 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-04-25T01:47:19.887 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-04-25T01:47:19.887 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-04-25T01:47:19.887 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-04-25T01:47:19.887 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-04-25T01:47:19.887 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-04-25T01:47:19.887 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-04-25T01:47:19.887 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-04-25T01:47:19.887 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-04-25T01:47:19.887 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-04-25T01:47:19.887 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-04-25T01:47:19.887 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-04-25T01:47:19.887 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-04-25T01:47:19.888 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-04-25T01:47:19.888 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-04-25T01:47:19.888 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-04-25T01:47:19.888 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-04-25T01:47:19.888 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-04-25T01:47:19.888 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-04-25T01:47:19.888 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-04-25T01:47:19.888 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-04-25T01:47:19.888 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-04-25T01:47:19.888 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-04-25T01:47:19.889 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-04-25T01:47:19.889 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-04-25T01:47:19.889 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-04-25T01:47:19.889 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-04-25T01:47:19.889 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-04-25T01:47:19.889 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-04-25T01:47:19.889 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-04-25T01:47:19.889 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-04-25T01:47:19.889 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-04-25T01:47:19.889 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-04-25T01:47:19.889 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-04-25T01:47:19.889 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-04-25T01:47:19.890 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-04-25T01:47:19.890 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-04-25T01:47:19.890 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-04-25T01:47:19.890 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-04-25T01:47:19.890 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-04-25T01:47:19.890 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-04-25T01:47:19.890 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-04-25T01:47:19.890 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-04-25T01:47:19.890 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-04-25T01:47:19.890 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-04-25T01:47:19.890 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-04-25T01:47:19.890 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-04-25T01:47:19.890 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-04-25T01:47:19.891 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-04-25T01:47:19.891 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-04-25T01:47:19.891 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-04-25T01:47:19.891 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-04-25T01:47:19.891 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-04-25T01:47:19.891 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-04-25T01:47:19.891 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-04-25T01:47:19.891 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-04-25T01:47:19.891 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-04-25T01:47:19.891 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-04-25T01:47:19.891 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-04-25T01:47:19.891 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-04-25T01:47:19.891 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-04-25T01:47:19.891 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-04-25T01:47:19.892 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-04-25T01:47:19.892 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-04-25T01:47:19.892 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-04-25T01:47:19.892 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-04-25T01:47:19.892 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-04-25T01:47:20.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:20 smithi152 ceph-mon[32465]: pgmap v415: 129 pgs: 129 active+clean; 340 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 91 MiB/s rd, 880 KiB/s wr, 297 op/s 2024-04-25T01:47:20.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:20 smithi196 ceph-mon[32978]: pgmap v415: 129 pgs: 129 active+clean; 340 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 91 MiB/s rd, 880 KiB/s wr, 297 op/s 2024-04-25T01:47:20.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:20 smithi078 ceph-mon[29469]: pgmap v415: 129 pgs: 129 active+clean; 340 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 91 MiB/s rd, 880 KiB/s wr, 297 op/s 2024-04-25T01:47:20.702 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-04-25T01:47:20.702 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-04-25T01:47:20.702 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-04-25T01:47:20.703 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-04-25T01:47:20.704 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-04-25T01:47:20.704 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-04-25T01:47:20.704 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-04-25T01:47:20.704 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-04-25T01:47:20.704 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-04-25T01:47:20.704 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-04-25T01:47:20.704 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-04-25T01:47:20.704 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-04-25T01:47:20.704 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-04-25T01:47:20.704 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-04-25T01:47:20.704 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-04-25T01:47:20.704 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-04-25T01:47:20.704 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-04-25T01:47:20.704 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-04-25T01:47:20.705 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-04-25T01:47:20.706 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-04-25T01:47:20.706 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-04-25T01:47:20.706 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-04-25T01:47:20.706 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-04-25T01:47:20.706 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-04-25T01:47:20.706 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-04-25T01:47:20.706 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-04-25T01:47:20.706 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-04-25T01:47:20.706 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-04-25T01:47:20.706 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-04-25T01:47:20.706 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-04-25T01:47:20.706 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-04-25T01:47:20.706 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-04-25T01:47:20.706 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-04-25T01:47:20.707 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-04-25T01:47:20.707 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-04-25T01:47:20.707 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-04-25T01:47:20.707 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-04-25T01:47:20.707 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-04-25T01:47:20.707 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-04-25T01:47:20.707 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-04-25T01:47:20.707 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-04-25T01:47:20.707 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-04-25T01:47:20.707 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-04-25T01:47:20.707 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-04-25T01:47:20.707 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-04-25T01:47:20.707 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-04-25T01:47:20.707 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-04-25T01:47:20.708 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-04-25T01:47:20.708 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-04-25T01:47:20.708 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-04-25T01:47:20.708 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-04-25T01:47:20.708 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-04-25T01:47:20.708 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-04-25T01:47:21.117 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-04-25T01:47:21.117 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-04-25T01:47:21.117 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-04-25T01:47:21.117 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-04-25T01:47:21.117 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-04-25T01:47:21.117 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-04-25T01:47:21.117 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-04-25T01:47:21.117 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-04-25T01:47:21.118 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-04-25T01:47:21.118 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-04-25T01:47:21.118 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-04-25T01:47:21.118 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-04-25T01:47:21.118 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-04-25T01:47:21.118 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-04-25T01:47:21.118 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-04-25T01:47:21.118 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-04-25T01:47:21.118 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-04-25T01:47:21.119 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-04-25T01:47:21.119 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-04-25T01:47:21.119 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-04-25T01:47:21.119 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-04-25T01:47:21.119 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-04-25T01:47:21.119 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-04-25T01:47:21.119 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-04-25T01:47:21.119 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-04-25T01:47:21.119 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-04-25T01:47:21.119 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-04-25T01:47:21.120 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-04-25T01:47:21.120 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-04-25T01:47:21.120 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-04-25T01:47:21.120 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-04-25T01:47:21.120 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-04-25T01:47:21.120 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-04-25T01:47:21.120 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-04-25T01:47:21.120 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-04-25T01:47:21.120 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-04-25T01:47:21.120 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-04-25T01:47:21.121 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-04-25T01:47:21.121 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-04-25T01:47:21.121 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-04-25T01:47:21.121 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-04-25T01:47:21.121 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-04-25T01:47:21.121 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-04-25T01:47:21.121 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-04-25T01:47:21.121 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-04-25T01:47:21.121 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-04-25T01:47:21.121 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-04-25T01:47:21.122 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-04-25T01:47:21.122 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-04-25T01:47:21.122 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-04-25T01:47:21.122 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-04-25T01:47:21.122 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-04-25T01:47:21.122 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-04-25T01:47:21.122 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-04-25T01:47:21.122 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-04-25T01:47:21.122 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-04-25T01:47:21.123 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-04-25T01:47:21.123 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-04-25T01:47:21.123 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-04-25T01:47:21.123 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-04-25T01:47:21.123 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-04-25T01:47:21.123 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-04-25T01:47:21.123 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-04-25T01:47:21.123 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-04-25T01:47:21.123 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-04-25T01:47:21.123 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-04-25T01:47:21.124 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-04-25T01:47:21.124 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-04-25T01:47:21.124 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-04-25T01:47:21.124 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-04-25T01:47:21.124 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-04-25T01:47:21.124 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-04-25T01:47:21.124 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-04-25T01:47:21.124 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-04-25T01:47:21.124 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-04-25T01:47:21.124 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-04-25T01:47:21.125 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-04-25T01:47:21.125 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-04-25T01:47:21.125 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-04-25T01:47:21.125 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-04-25T01:47:21.125 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-04-25T01:47:21.972 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-04-25T01:47:21.972 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-04-25T01:47:21.972 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-04-25T01:47:21.972 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-04-25T01:47:21.972 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-04-25T01:47:21.972 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-04-25T01:47:21.972 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-04-25T01:47:21.972 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-04-25T01:47:21.972 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-04-25T01:47:21.972 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-04-25T01:47:21.972 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-04-25T01:47:21.973 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-04-25T01:47:21.973 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-04-25T01:47:21.973 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-04-25T01:47:21.973 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-04-25T01:47:21.973 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-04-25T01:47:21.973 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-04-25T01:47:21.973 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-04-25T01:47:21.973 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-04-25T01:47:21.973 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-04-25T01:47:21.973 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-04-25T01:47:21.973 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-04-25T01:47:21.973 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-04-25T01:47:21.973 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-04-25T01:47:21.973 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-04-25T01:47:21.974 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-04-25T01:47:21.974 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-04-25T01:47:21.974 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-04-25T01:47:21.974 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-04-25T01:47:21.974 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-04-25T01:47:21.974 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-04-25T01:47:21.974 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-04-25T01:47:21.974 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-04-25T01:47:21.974 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-04-25T01:47:21.974 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-04-25T01:47:21.974 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-04-25T01:47:21.974 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-04-25T01:47:21.974 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-04-25T01:47:21.974 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-04-25T01:47:21.975 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-04-25T01:47:21.976 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-04-25T01:47:21.976 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-04-25T01:47:21.976 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-04-25T01:47:21.976 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-04-25T01:47:21.976 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-04-25T01:47:21.976 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-04-25T01:47:21.976 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-04-25T01:47:21.976 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-04-25T01:47:21.976 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-04-25T01:47:21.976 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-04-25T01:47:21.976 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-04-25T01:47:21.976 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-04-25T01:47:21.976 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-04-25T01:47:21.976 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-04-25T01:47:21.977 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-04-25T01:47:21.977 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-04-25T01:47:21.977 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-04-25T01:47:21.977 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-04-25T01:47:21.977 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-04-25T01:47:21.977 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-04-25T01:47:21.977 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-04-25T01:47:21.977 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-04-25T01:47:21.977 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-04-25T01:47:21.977 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-04-25T01:47:21.977 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-04-25T01:47:21.977 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-04-25T01:47:21.977 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-04-25T01:47:22.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:22 smithi152 ceph-mon[32465]: pgmap v416: 129 pgs: 129 active+clean; 259 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 107 MiB/s rd, 1001 KiB/s wr, 352 op/s 2024-04-25T01:47:22.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:22 smithi196 ceph-mon[32978]: pgmap v416: 129 pgs: 129 active+clean; 259 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 107 MiB/s rd, 1001 KiB/s wr, 352 op/s 2024-04-25T01:47:22.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:22 smithi078 ceph-mon[29469]: pgmap v416: 129 pgs: 129 active+clean; 259 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 107 MiB/s rd, 1001 KiB/s wr, 352 op/s 2024-04-25T01:47:23.169 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-04-25T01:47:23.169 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-04-25T01:47:23.169 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-04-25T01:47:23.169 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-04-25T01:47:23.169 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-04-25T01:47:23.169 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-04-25T01:47:23.169 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-04-25T01:47:23.169 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-04-25T01:47:23.170 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-04-25T01:47:23.170 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-04-25T01:47:23.170 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-04-25T01:47:23.170 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-04-25T01:47:23.170 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-04-25T01:47:23.170 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-04-25T01:47:23.170 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-04-25T01:47:23.170 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-04-25T01:47:23.170 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-04-25T01:47:23.170 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-04-25T01:47:23.170 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-04-25T01:47:23.170 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-04-25T01:47:23.170 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-04-25T01:47:23.171 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-04-25T01:47:23.172 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-04-25T01:47:23.172 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-04-25T01:47:23.172 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-04-25T01:47:23.172 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-04-25T01:47:23.172 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-04-25T01:47:23.172 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-04-25T01:47:23.172 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-04-25T01:47:23.172 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-04-25T01:47:23.172 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-04-25T01:47:23.172 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-04-25T01:47:23.172 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-04-25T01:47:23.172 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-04-25T01:47:23.172 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-04-25T01:47:23.172 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-04-25T01:47:23.173 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-04-25T01:47:23.174 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-04-25T01:47:23.762 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-04-25T01:47:23.762 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-04-25T01:47:23.763 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-04-25T01:47:23.763 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-04-25T01:47:23.763 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-04-25T01:47:23.763 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-04-25T01:47:23.763 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-04-25T01:47:23.764 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-04-25T01:47:23.764 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-04-25T01:47:23.765 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-04-25T01:47:23.765 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-04-25T01:47:23.765 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-04-25T01:47:23.765 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-04-25T01:47:23.767 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-04-25T01:47:23.767 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-04-25T01:47:23.767 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-04-25T01:47:23.767 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-04-25T01:47:23.767 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-04-25T01:47:23.767 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-04-25T01:47:23.767 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-04-25T01:47:23.768 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-04-25T01:47:23.768 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-04-25T01:47:23.768 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-04-25T01:47:23.768 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-04-25T01:47:23.768 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-04-25T01:47:23.768 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-04-25T01:47:23.768 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-04-25T01:47:23.769 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-04-25T01:47:23.769 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-04-25T01:47:23.769 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-04-25T01:47:23.769 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-04-25T01:47:23.769 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-04-25T01:47:23.769 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-04-25T01:47:23.769 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-04-25T01:47:23.769 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-04-25T01:47:23.769 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-04-25T01:47:23.769 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-04-25T01:47:23.770 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-04-25T01:47:23.770 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-04-25T01:47:23.770 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-04-25T01:47:23.770 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-04-25T01:47:23.770 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-04-25T01:47:23.770 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-04-25T01:47:23.770 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-04-25T01:47:23.770 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-04-25T01:47:23.770 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-04-25T01:47:23.771 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-04-25T01:47:23.771 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-04-25T01:47:23.771 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-04-25T01:47:23.771 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-04-25T01:47:23.771 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-04-25T01:47:23.771 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-04-25T01:47:23.771 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-04-25T01:47:23.771 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-04-25T01:47:23.771 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-04-25T01:47:23.771 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-04-25T01:47:23.771 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-04-25T01:47:23.772 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-04-25T01:47:23.772 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-04-25T01:47:23.772 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-04-25T01:47:23.772 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-04-25T01:47:23.772 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-04-25T01:47:23.772 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-04-25T01:47:23.772 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-04-25T01:47:23.772 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-04-25T01:47:23.772 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-04-25T01:47:23.772 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-04-25T01:47:23.772 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-04-25T01:47:23.772 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-04-25T01:47:23.773 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-04-25T01:47:23.773 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-04-25T01:47:23.773 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-04-25T01:47:23.773 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-04-25T01:47:23.773 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-04-25T01:47:23.773 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-04-25T01:47:23.773 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-04-25T01:47:23.773 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-04-25T01:47:23.773 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-04-25T01:47:23.773 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-04-25T01:47:23.774 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-04-25T01:47:23.774 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-04-25T01:47:24.194 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-04-25T01:47:24.195 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-04-25T01:47:24.195 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-04-25T01:47:24.195 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-04-25T01:47:24.195 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-04-25T01:47:24.195 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-04-25T01:47:24.195 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-04-25T01:47:24.195 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-04-25T01:47:24.195 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-04-25T01:47:24.195 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-04-25T01:47:24.195 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-04-25T01:47:24.195 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-04-25T01:47:24.195 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-04-25T01:47:24.195 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-04-25T01:47:24.196 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-04-25T01:47:24.196 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-04-25T01:47:24.196 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-04-25T01:47:24.196 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-04-25T01:47:24.196 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-04-25T01:47:24.196 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-04-25T01:47:24.196 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-04-25T01:47:24.196 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-04-25T01:47:24.196 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-04-25T01:47:24.196 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-04-25T01:47:24.196 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-04-25T01:47:24.196 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-04-25T01:47:24.196 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-04-25T01:47:24.196 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-04-25T01:47:24.197 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-04-25T01:47:24.198 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-04-25T01:47:24.198 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-04-25T01:47:24.198 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-04-25T01:47:24.198 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-04-25T01:47:24.198 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-04-25T01:47:24.198 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-04-25T01:47:24.198 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-04-25T01:47:24.198 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-04-25T01:47:24.198 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-04-25T01:47:24.198 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-04-25T01:47:24.199 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-04-25T01:47:24.199 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-04-25T01:47:24.199 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-04-25T01:47:24.199 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-04-25T01:47:24.199 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-04-25T01:47:24.199 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-04-25T01:47:24.199 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-04-25T01:47:24.199 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-04-25T01:47:24.199 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-04-25T01:47:24.199 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-04-25T01:47:24.200 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-04-25T01:47:24.201 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-04-25T01:47:24.201 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-04-25T01:47:24.201 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-04-25T01:47:24.268 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-04-25T01:47:24.268 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-04-25T01:47:24.269 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-04-25T01:47:24.269 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-04-25T01:47:24.269 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-04-25T01:47:24.269 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-04-25T01:47:24.269 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-04-25T01:47:24.269 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-04-25T01:47:24.269 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-04-25T01:47:24.269 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-04-25T01:47:24.269 INFO:tasks.workunit.client.0.smithi078.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-04-25T01:47:24.269 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-04-25T01:47:24.269 INFO:tasks.workunit.client.0.smithi078.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-04-25T01:47:24.269 INFO:tasks.workunit.client.0.smithi078.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-04-25T01:47:24.270 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T01:47:24.270 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T01:47:24.290 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:24 smithi078 ceph-mon[29469]: pgmap v417: 129 pgs: 129 active+clean; 223 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 932 KiB/s wr, 315 op/s 2024-04-25T01:47:24.298 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-04-25T01:47:24.299 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T01:47:24.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:24 smithi152 ceph-mon[32465]: pgmap v417: 129 pgs: 129 active+clean; 223 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 932 KiB/s wr, 315 op/s 2024-04-25T01:47:24.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:24 smithi196 ceph-mon[32978]: pgmap v417: 129 pgs: 129 active+clean; 223 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 932 KiB/s wr, 315 op/s 2024-04-25T01:47:24.622 DEBUG:teuthology.parallel:result is None 2024-04-25T01:47:24.622 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T01:47:24.657 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T01:47:24.658 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T01:47:24.670 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:47:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.800+0000 7f21246fc640 1 -- 172.21.15.78:0/735480746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c0ffa60 msgr2=0x7f211c0ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.800+0000 7f21246fc640 1 --2- 172.21.15.78:0/735480746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c0ffa60 0x7f211c0ffe80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2118009cd0 tx=0x7f211802a8e0 comp rx=0 tx=0).stop 2024-04-25T01:47:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.800+0000 7f21246fc640 1 -- 172.21.15.78:0/735480746 shutdown_connections 2024-04-25T01:47:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.800+0000 7f21246fc640 1 --2- 172.21.15.78:0/735480746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f211c1033f0 0x7f211c109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.800+0000 7f21246fc640 1 --2- 172.21.15.78:0/735480746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f211c102a30 0x7f211c102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.800+0000 7f21246fc640 1 --2- 172.21.15.78:0/735480746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c0ffa60 0x7f211c0ffe80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.800+0000 7f21246fc640 1 -- 172.21.15.78:0/735480746 >> 172.21.15.78:0/735480746 conn(0x7f211c0fb3b0 msgr2=0x7f211c0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.800+0000 7f21246fc640 1 -- 172.21.15.78:0/735480746 shutdown_connections 2024-04-25T01:47:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.800+0000 7f21246fc640 1 -- 172.21.15.78:0/735480746 wait complete. 2024-04-25T01:47:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f21246fc640 1 Processor -- start 2024-04-25T01:47:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f21246fc640 1 -- start start 2024-04-25T01:47:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f21246fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f211c0ffa60 0x7f211c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f21246fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f211c102a30 0x7f211c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f21246fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c1033f0 0x7f211c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f21246fc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f211c10d430 con 0x7f211c1033f0 2024-04-25T01:47:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f21246fc640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f211c10d2b0 con 0x7f211c0ffa60 2024-04-25T01:47:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f21246fc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f211c10d5b0 con 0x7f211c102a30 2024-04-25T01:47:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f2122c99640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c1033f0 0x7f211c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f2122c99640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c1033f0 0x7f211c19f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52480/0 (socket says 172.21.15.78:52480) 2024-04-25T01:47:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f2122c99640 1 -- 172.21.15.78:0/904855544 learned_addr learned my addr 172.21.15.78:0/904855544 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:47:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f2121c97640 1 --2- 172.21.15.78:0/904855544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f211c102a30 0x7f211c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f2122498640 1 --2- 172.21.15.78:0/904855544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f211c0ffa60 0x7f211c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f2122c99640 1 -- 172.21.15.78:0/904855544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f211c102a30 msgr2=0x7f211c198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f2122c99640 1 --2- 172.21.15.78:0/904855544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f211c102a30 0x7f211c198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f2122c99640 1 -- 172.21.15.78:0/904855544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f211c0ffa60 msgr2=0x7f211c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f2122c99640 1 --2- 172.21.15.78:0/904855544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f211c0ffa60 0x7f211c197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.801+0000 7f2122c99640 1 -- 172.21.15.78:0/904855544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211c19f750 con 0x7f211c1033f0 2024-04-25T01:47:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.802+0000 7f2122c99640 1 --2- 172.21.15.78:0/904855544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c1033f0 0x7f211c19f0e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f211400e970 tx=0x7f211400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.802+0000 7f21137fe640 1 -- 172.21.15.78:0/904855544 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211400ccc0 con 0x7f211c1033f0 2024-04-25T01:47:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.802+0000 7f21246fc640 1 -- 172.21.15.78:0/904855544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211c19fa40 con 0x7f211c1033f0 2024-04-25T01:47:24.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.802+0000 7f21137fe640 1 -- 172.21.15.78:0/904855544 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f211400ce60 con 0x7f211c1033f0 2024-04-25T01:47:24.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.802+0000 7f21137fe640 1 -- 172.21.15.78:0/904855544 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114010490 con 0x7f211c1033f0 2024-04-25T01:47:24.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.802+0000 7f21246fc640 1 -- 172.21.15.78:0/904855544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f211c19ff90 con 0x7f211c1033f0 2024-04-25T01:47:24.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.803+0000 7f21246fc640 1 -- 172.21.15.78:0/904855544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f211c108c30 con 0x7f211c1033f0 2024-04-25T01:47:24.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.804+0000 7f21137fe640 1 -- 172.21.15.78:0/904855544 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f2114005020 con 0x7f211c1033f0 2024-04-25T01:47:24.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.804+0000 7f21137fe640 1 --2- 172.21.15.78:0/904855544 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f21080a2b90 0x7f21080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:24.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.804+0000 7f2122498640 1 --2- 172.21.15.78:0/904855544 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f21080a2b90 0x7f21080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:24.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.804+0000 7f21137fe640 1 -- 172.21.15.78:0/904855544 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f21140cf940 con 0x7f211c1033f0 2024-04-25T01:47:24.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.805+0000 7f2122498640 1 --2- 172.21.15.78:0/904855544 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f21080a2b90 0x7f21080a5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2118006010 tx=0x7f2118000f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:24.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.809+0000 7f21137fe640 1 -- 172.21.15.78:0/904855544 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f211409c4c0 con 0x7f211c1033f0 2024-04-25T01:47:24.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.985+0000 7f21246fc640 1 -- 172.21.15.78:0/904855544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f211c062340 con 0x7f211c1033f0 2024-04-25T01:47:24.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.987+0000 7f21137fe640 1 -- 172.21.15.78:0/904855544 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24844 (secure 0 0 0) 0x7f211409bdc0 con 0x7f211c1033f0 2024-04-25T01:47:24.983 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:47:24.983 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":94,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:45:18.989934+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:47:24.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.989+0000 7f21246fc640 1 -- 172.21.15.78:0/904855544 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f21080a2b90 msgr2=0x7f21080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:24.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.989+0000 7f21246fc640 1 --2- 172.21.15.78:0/904855544 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f21080a2b90 0x7f21080a5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2118006010 tx=0x7f2118000f40 comp rx=0 tx=0).stop 2024-04-25T01:47:24.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.989+0000 7f21246fc640 1 -- 172.21.15.78:0/904855544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c1033f0 msgr2=0x7f211c19f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:24.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.989+0000 7f21246fc640 1 --2- 172.21.15.78:0/904855544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c1033f0 0x7f211c19f0e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f211400e970 tx=0x7f211400ed70 comp rx=0 tx=0).stop 2024-04-25T01:47:24.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.989+0000 7f21246fc640 1 -- 172.21.15.78:0/904855544 shutdown_connections 2024-04-25T01:47:24.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.989+0000 7f21246fc640 1 --2- 172.21.15.78:0/904855544 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f21080a2b90 0x7f21080a5050 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:24.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.989+0000 7f21246fc640 1 --2- 172.21.15.78:0/904855544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f211c1033f0 0x7f211c19f0e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:24.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.989+0000 7f21246fc640 1 --2- 172.21.15.78:0/904855544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f211c102a30 0x7f211c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:24.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.989+0000 7f21246fc640 1 --2- 172.21.15.78:0/904855544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f211c0ffa60 0x7f211c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:24.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.989+0000 7f21246fc640 1 -- 172.21.15.78:0/904855544 >> 172.21.15.78:0/904855544 conn(0x7f211c0fb3b0 msgr2=0x7f211c105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:24.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.989+0000 7f21246fc640 1 -- 172.21.15.78:0/904855544 shutdown_connections 2024-04-25T01:47:24.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:24.989+0000 7f21246fc640 1 -- 172.21.15.78:0/904855544 wait complete. 2024-04-25T01:47:24.996 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:38:39.136418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:43:15.613881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:43:16.600270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:43:19.650457+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:47:24.997 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:47:25.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 -- 172.21.15.78:0/2748308880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f264c100910 msgr2=0x7f264c100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 --2- 172.21.15.78:0/2748308880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f264c100910 0x7f264c100d90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f263c009930 tx=0x7f263c02a420 comp rx=0 tx=0).stop 2024-04-25T01:47:25.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 -- 172.21.15.78:0/2748308880 shutdown_connections 2024-04-25T01:47:25.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 --2- 172.21.15.78:0/2748308880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f264c1012d0 0x7f264c108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 --2- 172.21.15.78:0/2748308880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f264c100910 0x7f264c100d90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 --2- 172.21.15.78:0/2748308880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f264c101c70 0x7f264c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 -- 172.21.15.78:0/2748308880 >> 172.21.15.78:0/2748308880 conn(0x7f264c0fb380 msgr2=0x7f264c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:25.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 -- 172.21.15.78:0/2748308880 shutdown_connections 2024-04-25T01:47:25.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 -- 172.21.15.78:0/2748308880 wait complete. 2024-04-25T01:47:25.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 Processor -- start 2024-04-25T01:47:25.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 -- start start 2024-04-25T01:47:25.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f264c100910 0x7f264c196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:25.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f264c1012d0 0x7f264c196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:25.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f264c101c70 0x7f264c19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:25.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f264c10c330 con 0x7f264c100910 2024-04-25T01:47:25.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f264c10c1b0 con 0x7f264c101c70 2024-04-25T01:47:25.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.106+0000 7f26540d4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f264c10c4b0 con 0x7f264c1012d0 2024-04-25T01:47:25.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.107+0000 7f2651e70640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f264c100910 0x7f264c196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:25.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.107+0000 7f2651e70640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f264c100910 0x7f264c196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52506/0 (socket says 172.21.15.78:52506) 2024-04-25T01:47:25.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.107+0000 7f2651e70640 1 -- 172.21.15.78:0/542583800 learned_addr learned my addr 172.21.15.78:0/542583800 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:47:25.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.107+0000 7f2651e70640 1 -- 172.21.15.78:0/542583800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f264c1012d0 msgr2=0x7f264c196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.107+0000 7f2651e70640 1 --2- 172.21.15.78:0/542583800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f264c1012d0 0x7f264c196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.107+0000 7f2651e70640 1 -- 172.21.15.78:0/542583800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f264c101c70 msgr2=0x7f264c19df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.107+0000 7f2651e70640 1 --2- 172.21.15.78:0/542583800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f264c101c70 0x7f264c19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.107+0000 7f2651e70640 1 -- 172.21.15.78:0/542583800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c19e570 con 0x7f264c100910 2024-04-25T01:47:25.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.107+0000 7f2651e70640 1 --2- 172.21.15.78:0/542583800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f264c100910 0x7f264c196950 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f264800c970 tx=0x7f264800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:25.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.107+0000 7f2642ffd640 1 -- 172.21.15.78:0/542583800 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648007c10 con 0x7f264c100910 2024-04-25T01:47:25.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.107+0000 7f26540d4640 1 -- 172.21.15.78:0/542583800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c19e860 con 0x7f264c100910 2024-04-25T01:47:25.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.107+0000 7f2642ffd640 1 -- 172.21.15.78:0/542583800 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2648007db0 con 0x7f264c100910 2024-04-25T01:47:25.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.107+0000 7f2642ffd640 1 -- 172.21.15.78:0/542583800 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264800f8a0 con 0x7f264c100910 2024-04-25T01:47:25.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.107+0000 7f26540d4640 1 -- 172.21.15.78:0/542583800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c19ee00 con 0x7f264c100910 2024-04-25T01:47:25.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.108+0000 7f26540d4640 1 -- 172.21.15.78:0/542583800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264c107a50 con 0x7f264c100910 2024-04-25T01:47:25.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.109+0000 7f2642ffd640 1 -- 172.21.15.78:0/542583800 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f264800fa80 con 0x7f264c100910 2024-04-25T01:47:25.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.110+0000 7f2642ffd640 1 --2- 172.21.15.78:0/542583800 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f26380a2b90 0x7f26380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:25.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.110+0000 7f265166f640 1 --2- 172.21.15.78:0/542583800 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f26380a2b90 0x7f26380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:25.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.110+0000 7f2642ffd640 1 -- 172.21.15.78:0/542583800 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f26480cf820 con 0x7f264c100910 2024-04-25T01:47:25.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.110+0000 7f265166f640 1 --2- 172.21.15.78:0/542583800 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f26380a2b90 0x7f26380a5050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f263c02a940 tx=0x7f263c031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:25.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.114+0000 7f2642ffd640 1 -- 172.21.15.78:0/542583800 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f264809c430 con 0x7f264c100910 2024-04-25T01:47:25.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:25 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/904855544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:47:25.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.290+0000 7f26540d4640 1 -- 172.21.15.78:0/542583800 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f264c0623e0 con 0x7f264c100910 2024-04-25T01:47:25.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.291+0000 7f2642ffd640 1 -- 172.21.15.78:0/542583800 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f264809bd30 con 0x7f264c100910 2024-04-25T01:47:25.286 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T01:47:25.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.293+0000 7f26540d4640 1 -- 172.21.15.78:0/542583800 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f26380a2b90 msgr2=0x7f26380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.293+0000 7f26540d4640 1 --2- 172.21.15.78:0/542583800 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f26380a2b90 0x7f26380a5050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f263c02a940 tx=0x7f263c031400 comp rx=0 tx=0).stop 2024-04-25T01:47:25.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.293+0000 7f26540d4640 1 -- 172.21.15.78:0/542583800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f264c100910 msgr2=0x7f264c196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.293+0000 7f26540d4640 1 --2- 172.21.15.78:0/542583800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f264c100910 0x7f264c196950 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f264800c970 tx=0x7f264800ce30 comp rx=0 tx=0).stop 2024-04-25T01:47:25.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.293+0000 7f26540d4640 1 -- 172.21.15.78:0/542583800 shutdown_connections 2024-04-25T01:47:25.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.293+0000 7f26540d4640 1 --2- 172.21.15.78:0/542583800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f264c101c70 0x7f264c19df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.293+0000 7f26540d4640 1 --2- 172.21.15.78:0/542583800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f264c1012d0 0x7f264c196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.293+0000 7f26540d4640 1 --2- 172.21.15.78:0/542583800 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f26380a2b90 0x7f26380a5050 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.293+0000 7f26540d4640 1 --2- 172.21.15.78:0/542583800 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f264c100910 0x7f264c196950 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.293+0000 7f26540d4640 1 -- 172.21.15.78:0/542583800 >> 172.21.15.78:0/542583800 conn(0x7f264c0fb380 msgr2=0x7f264c104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:25.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.293+0000 7f26540d4640 1 -- 172.21.15.78:0/542583800 shutdown_connections 2024-04-25T01:47:25.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.293+0000 7f26540d4640 1 -- 172.21.15.78:0/542583800 wait complete. 2024-04-25T01:47:25.300 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:47:25.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:25 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/904855544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:47:25.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:25 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/904855544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:47:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 -- 172.21.15.78:0/4144487829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead8102230 msgr2=0x7fead8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 --2- 172.21.15.78:0/4144487829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead8102230 0x7fead8108ac0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fead0009ca0 tx=0x7fead002a8c0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 -- 172.21.15.78:0/4144487829 shutdown_connections 2024-04-25T01:47:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 --2- 172.21.15.78:0/4144487829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead8102230 0x7fead8108ac0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 --2- 172.21.15.78:0/4144487829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead8101870 0x7fead8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 --2- 172.21.15.78:0/4144487829 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fead8100680 0x7fead8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 -- 172.21.15.78:0/4144487829 >> 172.21.15.78:0/4144487829 conn(0x7fead80fbbf0 msgr2=0x7fead80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 -- 172.21.15.78:0/4144487829 shutdown_connections 2024-04-25T01:47:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 -- 172.21.15.78:0/4144487829 wait complete. 2024-04-25T01:47:25.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 Processor -- start 2024-04-25T01:47:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 -- start start 2024-04-25T01:47:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead8100680 0x7fead8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead8101870 0x7fead8197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fead8102230 0x7fead819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fead810d420 con 0x7fead8101870 2024-04-25T01:47:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fead810d2a0 con 0x7fead8102230 2024-04-25T01:47:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadf94f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fead810d5a0 con 0x7fead8100680 2024-04-25T01:47:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.406+0000 7feadceea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead8101870 0x7fead8197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feadceea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead8101870 0x7fead8197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52510/0 (socket says 172.21.15.78:52510) 2024-04-25T01:47:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feadceea640 1 -- 172.21.15.78:0/2213752734 learned_addr learned my addr 172.21.15.78:0/2213752734 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:47:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feadd6eb640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead8100680 0x7fead8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feadceea640 1 -- 172.21.15.78:0/2213752734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead8100680 msgr2=0x7fead8197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feadceea640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead8100680 0x7fead8197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feaddeec640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fead8102230 0x7fead819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feadceea640 1 -- 172.21.15.78:0/2213752734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fead8102230 msgr2=0x7fead819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feadceea640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fead8102230 0x7fead819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feadceea640 1 -- 172.21.15.78:0/2213752734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead819f6a0 con 0x7fead8101870 2024-04-25T01:47:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feaddeec640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fead8102230 0x7fead819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:47:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feadceea640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead8101870 0x7fead8197fc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7feac800c970 tx=0x7feac800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feace7fc640 1 -- 172.21.15.78:0/2213752734 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8007bb0 con 0x7fead8101870 2024-04-25T01:47:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feace7fc640 1 -- 172.21.15.78:0/2213752734 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac8007d50 con 0x7fead8101870 2024-04-25T01:47:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feadf94f640 1 -- 172.21.15.78:0/2213752734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead819f990 con 0x7fead8101870 2024-04-25T01:47:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feace7fc640 1 -- 172.21.15.78:0/2213752734 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8018800 con 0x7fead8101870 2024-04-25T01:47:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.407+0000 7feadf94f640 1 -- 172.21.15.78:0/2213752734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead819fee0 con 0x7fead8101870 2024-04-25T01:47:25.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.408+0000 7feadf94f640 1 -- 172.21.15.78:0/2213752734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead81079a0 con 0x7fead8101870 2024-04-25T01:47:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.409+0000 7feace7fc640 1 -- 172.21.15.78:0/2213752734 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7feac800f450 con 0x7fead8101870 2024-04-25T01:47:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.410+0000 7feace7fc640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7feac40a2c10 0x7feac40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.410+0000 7feadd6eb640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7feac40a2c10 0x7feac40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.410+0000 7feace7fc640 1 -- 172.21.15.78:0/2213752734 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7feac80cf480 con 0x7fead8101870 2024-04-25T01:47:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.410+0000 7feadd6eb640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7feac40a2c10 0x7feac40a50d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fead81016d0 tx=0x7fead4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:25.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.414+0000 7feace7fc640 1 -- 172.21.15.78:0/2213752734 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feac8014030 con 0x7fead8101870 2024-04-25T01:47:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.594+0000 7feadf94f640 1 -- 172.21.15.78:0/2213752734 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fead8062370 con 0x7fead8101870 2024-04-25T01:47:25.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.594+0000 7feace7fc640 1 -- 172.21.15.78:0/2213752734 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7feac801f2f0 con 0x7fead8101870 2024-04-25T01:47:25.590 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:47:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.596+0000 7feadf94f640 1 -- 172.21.15.78:0/2213752734 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7feac40a2c10 msgr2=0x7feac40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.596+0000 7feadf94f640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7feac40a2c10 0x7feac40a50d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fead81016d0 tx=0x7fead4005f90 comp rx=0 tx=0).stop 2024-04-25T01:47:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.596+0000 7feadf94f640 1 -- 172.21.15.78:0/2213752734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead8101870 msgr2=0x7fead8197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.596+0000 7feadf94f640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead8101870 0x7fead8197fc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7feac800c970 tx=0x7feac800ce30 comp rx=0 tx=0).stop 2024-04-25T01:47:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.596+0000 7feadf94f640 1 -- 172.21.15.78:0/2213752734 shutdown_connections 2024-04-25T01:47:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.596+0000 7feadf94f640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fead8102230 0x7fead819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.596+0000 7feadf94f640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7feac40a2c10 0x7feac40a50d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.596+0000 7feadf94f640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fead8101870 0x7fead8197fc0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.596+0000 7feadf94f640 1 --2- 172.21.15.78:0/2213752734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fead8100680 0x7fead8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.596+0000 7feadf94f640 1 -- 172.21.15.78:0/2213752734 >> 172.21.15.78:0/2213752734 conn(0x7fead80fbbf0 msgr2=0x7fead8104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.596+0000 7feadf94f640 1 -- 172.21.15.78:0/2213752734 shutdown_connections 2024-04-25T01:47:25.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.596+0000 7feadf94f640 1 -- 172.21.15.78:0/2213752734 wait complete. 2024-04-25T01:47:25.603 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:47:25.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.708+0000 7f75d4516640 1 -- 172.21.15.78:0/2752796979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75cc102980 msgr2=0x7f75cc102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.708+0000 7f75d4516640 1 --2- 172.21.15.78:0/2752796979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75cc102980 0x7f75cc102e00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009900 tx=0x7f75bc02a410 comp rx=0 tx=0).stop 2024-04-25T01:47:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.708+0000 7f75d4516640 1 -- 172.21.15.78:0/2752796979 shutdown_connections 2024-04-25T01:47:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.708+0000 7f75d4516640 1 --2- 172.21.15.78:0/2752796979 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75cc103340 0x7f75cc109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.708+0000 7f75d4516640 1 --2- 172.21.15.78:0/2752796979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75cc102980 0x7f75cc102e00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.708+0000 7f75d4516640 1 --2- 172.21.15.78:0/2752796979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75cc101790 0x7f75cc101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.708+0000 7f75d4516640 1 -- 172.21.15.78:0/2752796979 >> 172.21.15.78:0/2752796979 conn(0x7f75cc0fcd20 msgr2=0x7f75cc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.708+0000 7f75d4516640 1 -- 172.21.15.78:0/2752796979 shutdown_connections 2024-04-25T01:47:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.708+0000 7f75d4516640 1 -- 172.21.15.78:0/2752796979 wait complete. 2024-04-25T01:47:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d4516640 1 Processor -- start 2024-04-25T01:47:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d4516640 1 -- start start 2024-04-25T01:47:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d4516640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75cc101790 0x7f75cc0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:25.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d4516640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75cc102980 0x7f75cc073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d4516640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75cc103340 0x7f75cc074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d4516640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc10d410 con 0x7f75cc101790 2024-04-25T01:47:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d4516640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc10d290 con 0x7f75cc103340 2024-04-25T01:47:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d4516640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc10d590 con 0x7f75cc102980 2024-04-25T01:47:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d22b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75cc101790 0x7f75cc0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d22b2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75cc101790 0x7f75cc0754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52526/0 (socket says 172.21.15.78:52526) 2024-04-25T01:47:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d22b2640 1 -- 172.21.15.78:0/3744349217 learned_addr learned my addr 172.21.15.78:0/3744349217 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:47:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d1ab1640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75cc102980 0x7f75cc073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d22b2640 1 -- 172.21.15.78:0/3744349217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75cc102980 msgr2=0x7f75cc073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d2ab3640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75cc103340 0x7f75cc074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d22b2640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75cc102980 0x7f75cc073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d22b2640 1 -- 172.21.15.78:0/3744349217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75cc103340 msgr2=0x7f75cc074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d22b2640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75cc103340 0x7f75cc074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d22b2640 1 -- 172.21.15.78:0/3744349217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc074900 con 0x7f75cc101790 2024-04-25T01:47:25.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.709+0000 7f75d1ab1640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75cc102980 0x7f75cc073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:47:25.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.710+0000 7f75d22b2640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75cc101790 0x7f75cc0754f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f75c800c970 tx=0x7f75c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:25.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.710+0000 7f75c37fe640 1 -- 172.21.15.78:0/3744349217 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8007c10 con 0x7f75cc101790 2024-04-25T01:47:25.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.710+0000 7f75c37fe640 1 -- 172.21.15.78:0/3744349217 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75c8007db0 con 0x7f75cc101790 2024-04-25T01:47:25.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.710+0000 7f75d4516640 1 -- 172.21.15.78:0/3744349217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75cc1a1b80 con 0x7f75cc101790 2024-04-25T01:47:25.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.710+0000 7f75c37fe640 1 -- 172.21.15.78:0/3744349217 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c800f940 con 0x7f75cc101790 2024-04-25T01:47:25.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.710+0000 7f75d4516640 1 -- 172.21.15.78:0/3744349217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75cc1a20a0 con 0x7f75cc101790 2024-04-25T01:47:25.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.711+0000 7f75d4516640 1 -- 172.21.15.78:0/3744349217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75cc0689b0 con 0x7f75cc101790 2024-04-25T01:47:25.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.712+0000 7f75c37fe640 1 -- 172.21.15.78:0/3744349217 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f75c800fae0 con 0x7f75cc101790 2024-04-25T01:47:25.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.713+0000 7f75c37fe640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f75b80a2b90 0x7f75b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:25.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.713+0000 7f75d1ab1640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f75b80a2b90 0x7f75b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:25.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.713+0000 7f75c37fe640 1 -- 172.21.15.78:0/3744349217 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f75c80cf910 con 0x7f75cc101790 2024-04-25T01:47:25.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.713+0000 7f75d1ab1640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f75b80a2b90 0x7f75b80a5050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f75bc000c00 tx=0x7f75bc0315d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:25.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.717+0000 7f75c37fe640 1 -- 172.21.15.78:0/3744349217 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f75c809c520 con 0x7f75cc101790 2024-04-25T01:47:25.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.893+0000 7f75d4516640 1 -- 172.21.15.78:0/3744349217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f75cc0623e0 con 0x7f75cc101790 2024-04-25T01:47:25.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.893+0000 7f75c37fe640 1 -- 172.21.15.78:0/3744349217 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f75c809be20 con 0x7f75cc101790 2024-04-25T01:47:25.889 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T01:47:25.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.895+0000 7f75d4516640 1 -- 172.21.15.78:0/3744349217 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f75b80a2b90 msgr2=0x7f75b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.895+0000 7f75d4516640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f75b80a2b90 0x7f75b80a5050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f75bc000c00 tx=0x7f75bc0315d0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.895+0000 7f75d4516640 1 -- 172.21.15.78:0/3744349217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75cc101790 msgr2=0x7f75cc0754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:25.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.895+0000 7f75d4516640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75cc101790 0x7f75cc0754f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f75c800c970 tx=0x7f75c800ce30 comp rx=0 tx=0).stop 2024-04-25T01:47:25.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.895+0000 7f75d4516640 1 -- 172.21.15.78:0/3744349217 shutdown_connections 2024-04-25T01:47:25.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.895+0000 7f75d4516640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75cc103340 0x7f75cc074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.895+0000 7f75d4516640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75cc102980 0x7f75cc073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.895+0000 7f75d4516640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f75b80a2b90 0x7f75b80a5050 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.895+0000 7f75d4516640 1 --2- 172.21.15.78:0/3744349217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75cc101790 0x7f75cc0754f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:25.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.895+0000 7f75d4516640 1 -- 172.21.15.78:0/3744349217 >> 172.21.15.78:0/3744349217 conn(0x7f75cc0fcd20 msgr2=0x7f75cc105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:25.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.895+0000 7f75d4516640 1 -- 172.21.15.78:0/3744349217 shutdown_connections 2024-04-25T01:47:25.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:25.895+0000 7f75d4516640 1 -- 172.21.15.78:0/3744349217 wait complete. 2024-04-25T01:47:25.902 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:47:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.007+0000 7f7fd9322640 1 -- 172.21.15.78:0/300645001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fd4102980 msgr2=0x7f7fd4102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.007+0000 7f7fd9322640 1 --2- 172.21.15.78:0/300645001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fd4102980 0x7f7fd4102e00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009900 tx=0x7f7fc802a410 comp rx=0 tx=0).stop 2024-04-25T01:47:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.007+0000 7f7fd9322640 1 -- 172.21.15.78:0/300645001 shutdown_connections 2024-04-25T01:47:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.007+0000 7f7fd9322640 1 --2- 172.21.15.78:0/300645001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fd4103340 0x7f7fd4109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.007+0000 7f7fd9322640 1 --2- 172.21.15.78:0/300645001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fd4102980 0x7f7fd4102e00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.007+0000 7f7fd9322640 1 --2- 172.21.15.78:0/300645001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fd4101790 0x7f7fd4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.007+0000 7f7fd9322640 1 -- 172.21.15.78:0/300645001 >> 172.21.15.78:0/300645001 conn(0x7f7fd40fcd20 msgr2=0x7f7fd40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.007+0000 7f7fd9322640 1 -- 172.21.15.78:0/300645001 shutdown_connections 2024-04-25T01:47:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.007+0000 7f7fd9322640 1 -- 172.21.15.78:0/300645001 wait complete. 2024-04-25T01:47:26.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd9322640 1 Processor -- start 2024-04-25T01:47:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd9322640 1 -- start start 2024-04-25T01:47:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd9322640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fd4101790 0x7f7fd4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd9322640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fd4102980 0x7f7fd4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd9322640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fd4103340 0x7f7fd419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd9322640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd410d410 con 0x7f7fd4103340 2024-04-25T01:47:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd9322640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd410d290 con 0x7f7fd4101790 2024-04-25T01:47:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd9322640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd410d590 con 0x7f7fd4102980 2024-04-25T01:47:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fd4103340 0x7f7fd419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fd4103340 0x7f7fd419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52544/0 (socket says 172.21.15.78:52544) 2024-04-25T01:47:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd37fe640 1 -- 172.21.15.78:0/2063114750 learned_addr learned my addr 172.21.15.78:0/2063114750 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:47:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd2ffd640 1 --2- 172.21.15.78:0/2063114750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fd4101790 0x7f7fd4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd27fc640 1 --2- 172.21.15.78:0/2063114750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fd4102980 0x7f7fd4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd37fe640 1 -- 172.21.15.78:0/2063114750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fd4102980 msgr2=0x7f7fd4197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:26.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd37fe640 1 --2- 172.21.15.78:0/2063114750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fd4102980 0x7f7fd4197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd37fe640 1 -- 172.21.15.78:0/2063114750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fd4101790 msgr2=0x7f7fd4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd37fe640 1 --2- 172.21.15.78:0/2063114750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fd4101790 0x7f7fd4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.008+0000 7f7fd37fe640 1 -- 172.21.15.78:0/2063114750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd419f6d0 con 0x7f7fd4103340 2024-04-25T01:47:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.009+0000 7f7fd37fe640 1 --2- 172.21.15.78:0/2063114750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fd4103340 0x7f7fd419f060 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00eaa0 tx=0x7f7fcc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.009+0000 7f7fbbfff640 1 -- 172.21.15.78:0/2063114750 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc00cdb0 con 0x7f7fd4103340 2024-04-25T01:47:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.009+0000 7f7fd9322640 1 -- 172.21.15.78:0/2063114750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd419f9c0 con 0x7f7fd4103340 2024-04-25T01:47:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.009+0000 7f7fbbfff640 1 -- 172.21.15.78:0/2063114750 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc005020 con 0x7f7fd4103340 2024-04-25T01:47:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.009+0000 7f7fbbfff640 1 -- 172.21.15.78:0/2063114750 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0116d0 con 0x7f7fd4103340 2024-04-25T01:47:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.009+0000 7f7fd9322640 1 -- 172.21.15.78:0/2063114750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd419ff60 con 0x7f7fd4103340 2024-04-25T01:47:26.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.010+0000 7f7fd9322640 1 -- 172.21.15.78:0/2063114750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd4108b30 con 0x7f7fd4103340 2024-04-25T01:47:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.011+0000 7f7fbbfff640 1 -- 172.21.15.78:0/2063114750 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7fcc0054d0 con 0x7f7fd4103340 2024-04-25T01:47:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.012+0000 7f7fbbfff640 1 --2- 172.21.15.78:0/2063114750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7fbc0a2b90 0x7f7fbc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.012+0000 7f7fd2ffd640 1 --2- 172.21.15.78:0/2063114750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7fbc0a2b90 0x7f7fbc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.012+0000 7f7fbbfff640 1 -- 172.21.15.78:0/2063114750 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f7fcc0cf9a0 con 0x7f7fd4103340 2024-04-25T01:47:26.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.012+0000 7f7fd2ffd640 1 --2- 172.21.15.78:0/2063114750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7fbc0a2b90 0x7f7fbc0a5050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4005fd0 tx=0x7f7fc400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:26.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.015+0000 7f7fbbfff640 1 -- 172.21.15.78:0/2063114750 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7fcc016030 con 0x7f7fd4103340 2024-04-25T01:47:26.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.192+0000 7f7fd9322640 1 -- 172.21.15.78:0/2063114750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7fd402d300 con 0x7f7fd4103340 2024-04-25T01:47:26.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.192+0000 7f7fbbfff640 1 -- 172.21.15.78:0/2063114750 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f7fcc09c520 con 0x7f7fd4103340 2024-04-25T01:47:26.188 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:47:26.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.194+0000 7f7fd9322640 1 -- 172.21.15.78:0/2063114750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7fbc0a2b90 msgr2=0x7f7fbc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:26.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.194+0000 7f7fd9322640 1 --2- 172.21.15.78:0/2063114750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7fbc0a2b90 0x7f7fbc0a5050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4005fd0 tx=0x7f7fc400b410 comp rx=0 tx=0).stop 2024-04-25T01:47:26.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.194+0000 7f7fd9322640 1 -- 172.21.15.78:0/2063114750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fd4103340 msgr2=0x7f7fd419f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:26.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.194+0000 7f7fd9322640 1 --2- 172.21.15.78:0/2063114750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fd4103340 0x7f7fd419f060 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00eaa0 tx=0x7f7fcc00ef60 comp rx=0 tx=0).stop 2024-04-25T01:47:26.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.194+0000 7f7fd9322640 1 -- 172.21.15.78:0/2063114750 shutdown_connections 2024-04-25T01:47:26.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.194+0000 7f7fd9322640 1 --2- 172.21.15.78:0/2063114750 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7fbc0a2b90 0x7f7fbc0a5050 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.194+0000 7f7fd9322640 1 --2- 172.21.15.78:0/2063114750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7fd4103340 0x7f7fd419f060 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.194+0000 7f7fd9322640 1 --2- 172.21.15.78:0/2063114750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7fd4102980 0x7f7fd4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.194+0000 7f7fd9322640 1 --2- 172.21.15.78:0/2063114750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7fd4101790 0x7f7fd4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.194+0000 7f7fd9322640 1 -- 172.21.15.78:0/2063114750 >> 172.21.15.78:0/2063114750 conn(0x7f7fd40fcd20 msgr2=0x7f7fd4105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:26.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.194+0000 7f7fd9322640 1 -- 172.21.15.78:0/2063114750 shutdown_connections 2024-04-25T01:47:26.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.194+0000 7f7fd9322640 1 -- 172.21.15.78:0/2063114750 wait complete. 2024-04-25T01:47:26.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:26 smithi078 ceph-mon[29469]: pgmap v418: 129 pgs: 129 active+clean; 171 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 110 MiB/s rd, 926 KiB/s wr, 348 op/s 2024-04-25T01:47:26.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:26 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/542583800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:47:26.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:26 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2213752734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:47:26.200 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:26 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3744349217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:47:26.201 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T01:47:26.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.303+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/218082217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d341022b0 msgr2=0x7f5d34108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:26.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.303+0000 7f5d3c7d5640 1 --2- 172.21.15.78:0/218082217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d341022b0 0x7f5d34108b40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009ca0 tx=0x7f5d2c02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.303+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/218082217 shutdown_connections 2024-04-25T01:47:26.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.303+0000 7f5d3c7d5640 1 --2- 172.21.15.78:0/218082217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d341022b0 0x7f5d34108b40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.303+0000 7f5d3c7d5640 1 --2- 172.21.15.78:0/218082217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d341018f0 0x7f5d34101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.303+0000 7f5d3c7d5640 1 --2- 172.21.15.78:0/218082217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d34100700 0x7f5d34100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.303+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/218082217 >> 172.21.15.78:0/218082217 conn(0x7f5d340fbc90 msgr2=0x7f5d340fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:26.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.303+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/218082217 shutdown_connections 2024-04-25T01:47:26.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.303+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/218082217 wait complete. 2024-04-25T01:47:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3c7d5640 1 Processor -- start 2024-04-25T01:47:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3c7d5640 1 -- start start 2024-04-25T01:47:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3c7d5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34100700 0x7f5d3406a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3c7d5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d341018f0 0x7f5d3419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3c7d5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d341022b0 0x7f5d3419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3c7d5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3410c310 con 0x7f5d34100700 2024-04-25T01:47:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3c7d5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3410c190 con 0x7f5d341018f0 2024-04-25T01:47:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3c7d5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3410c490 con 0x7f5d341022b0 2024-04-25T01:47:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3a571640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34100700 0x7f5d3406a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:26.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3a571640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34100700 0x7f5d3406a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52556/0 (socket says 172.21.15.78:52556) 2024-04-25T01:47:26.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3a571640 1 -- 172.21.15.78:0/2711880251 learned_addr learned my addr 172.21.15.78:0/2711880251 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:47:26.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3ad72640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d341022b0 0x7f5d3419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:26.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3a571640 1 -- 172.21.15.78:0/2711880251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d341022b0 msgr2=0x7f5d3419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:26.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3a571640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d341022b0 0x7f5d3419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3a571640 1 -- 172.21.15.78:0/2711880251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d341018f0 msgr2=0x7f5d3419c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:26.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d39d70640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d341018f0 0x7f5d3419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:26.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3a571640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d341018f0 0x7f5d3419c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:26.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3a571640 1 -- 172.21.15.78:0/2711880251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3419f620 con 0x7f5d34100700 2024-04-25T01:47:26.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3ad72640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d341022b0 0x7f5d3419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:47:26.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d3a571640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34100700 0x7f5d3406a950 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000c970 tx=0x7f5d3000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:26.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.304+0000 7f5d2b7fe640 1 -- 172.21.15.78:0/2711880251 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30007c10 con 0x7f5d34100700 2024-04-25T01:47:26.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.305+0000 7f5d2b7fe640 1 -- 172.21.15.78:0/2711880251 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d30007db0 con 0x7f5d34100700 2024-04-25T01:47:26.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.305+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/2711880251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3419f910 con 0x7f5d34100700 2024-04-25T01:47:26.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.305+0000 7f5d2b7fe640 1 -- 172.21.15.78:0/2711880251 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3000f940 con 0x7f5d34100700 2024-04-25T01:47:26.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.305+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/2711880251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3419fe60 con 0x7f5d34100700 2024-04-25T01:47:26.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.305+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/2711880251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d34107950 con 0x7f5d34100700 2024-04-25T01:47:26.305 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:26 smithi152 ceph-mon[32465]: pgmap v418: 129 pgs: 129 active+clean; 171 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 110 MiB/s rd, 926 KiB/s wr, 348 op/s 2024-04-25T01:47:26.305 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:26 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/542583800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:47:26.306 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:26 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2213752734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:47:26.306 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:26 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3744349217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:47:26.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.307+0000 7f5d2b7fe640 1 -- 172.21.15.78:0/2711880251 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f5d3000fae0 con 0x7f5d34100700 2024-04-25T01:47:26.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.307+0000 7f5d2b7fe640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d200a2b90 0x7f5d200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.307+0000 7f5d39d70640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d200a2b90 0x7f5d200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.307+0000 7f5d2b7fe640 1 -- 172.21.15.78:0/2711880251 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f5d300cf8d0 con 0x7f5d34100700 2024-04-25T01:47:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.308+0000 7f5d39d70640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d200a2b90 0x7f5d200a5050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5d24005fd0 tx=0x7f5d24005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.311+0000 7f5d2b7fe640 1 -- 172.21.15.78:0/2711880251 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5d3009c4e0 con 0x7f5d34100700 2024-04-25T01:47:26.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:26 smithi196 ceph-mon[32978]: pgmap v418: 129 pgs: 129 active+clean; 171 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 110 MiB/s rd, 926 KiB/s wr, 348 op/s 2024-04-25T01:47:26.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:26 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/542583800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:47:26.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:26 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2213752734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:47:26.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:26 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3744349217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:47:26.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.533+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/2711880251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f5d34062370 con 0x7f5d34100700 2024-04-25T01:47:26.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.542+0000 7f5d2b7fe640 1 -- 172.21.15.78:0/2711880251 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740371 (secure 0 0 0) 0x7f5d3009bde0 con 0x7f5d34100700 2024-04-25T01:47:26.548 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:47:26.549 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:47:26.549 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 20, 2024-04-25T01:47:26.549 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.549 INFO:teuthology.orchestra.run.smithi078.stdout: "active_gid": 14148, 2024-04-25T01:47:26.549 INFO:teuthology.orchestra.run.smithi078.stdout: "active_name": "x", 2024-04-25T01:47:26.549 INFO:teuthology.orchestra.run.smithi078.stdout: "active_addrs": { 2024-04-25T01:47:26.549 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-04-25T01:47:26.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.550 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-04-25T01:47:26.550 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:6800", 2024-04-25T01:47:26.550 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2190983290 2024-04-25T01:47:26.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.550 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-04-25T01:47:26.550 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:6801", 2024-04-25T01:47:26.551 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2190983290 2024-04-25T01:47:26.551 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.551 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T01:47:26.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.551 INFO:teuthology.orchestra.run.smithi078.stdout: "active_addr": "172.21.15.78:6801/2190983290", 2024-04-25T01:47:26.551 INFO:teuthology.orchestra.run.smithi078.stdout: "active_change": "2024-04-25T01:36:16.812655+0000", 2024-04-25T01:47:26.551 INFO:teuthology.orchestra.run.smithi078.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T01:47:26.551 INFO:teuthology.orchestra.run.smithi078.stdout: "available": true, 2024-04-25T01:47:26.551 INFO:teuthology.orchestra.run.smithi078.stdout: "standbys": [ 2024-04-25T01:47:26.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.552 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 24109, 2024-04-25T01:47:26.552 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "z", 2024-04-25T01:47:26.552 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr_features": 4540701547738038271, 2024-04-25T01:47:26.552 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-04-25T01:47:26.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.552 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-04-25T01:47:26.552 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.553 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.553 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.553 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:26.553 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:26.553 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.553 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.553 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.553 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:26.553 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.554 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.554 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.554 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T01:47:26.554 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.554 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.554 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.554 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.555 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.555 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.555 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.555 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.555 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.555 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.555 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.556 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.556 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.556 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.556 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.556 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.556 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.556 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.556 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.557 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.557 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.557 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.557 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.557 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.557 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.557 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-04-25T01:47:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-04-25T01:47:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-04-25T01:47:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-04-25T01:47:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-04-25T01:47:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-04-25T01:47:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-04-25T01:47:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-04-25T01:47:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-04-25T01:47:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-04-25T01:47:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-04-25T01:47:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-04-25T01:47:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-04-25T01:47:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-04-25T01:47:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-04-25T01:47:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-04-25T01:47:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-04-25T01:47:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-04-25T01:47:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-04-25T01:47:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-04-25T01:47:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-04-25T01:47:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-04-25T01:47:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T01:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-04-25T01:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-04-25T01:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-04-25T01:47:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-04-25T01:47:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-04-25T01:47:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-04-25T01:47:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T01:47:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-04-25T01:47:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-04-25T01:47:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-04-25T01:47:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T01:47:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T01:47:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-04-25T01:47:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-04-25T01:47:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-04-25T01:47:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-04-25T01:47:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T01:47:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T01:47:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-04-25T01:47:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-04-25T01:47:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-04-25T01:47:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T01:47:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-04-25T01:47:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T01:47:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-04-25T01:47:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-04-25T01:47:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-04-25T01:47:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-04-25T01:47:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-04-25T01:47:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-04-25T01:47:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T01:47:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-04-25T01:47:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-04-25T01:47:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-04-25T01:47:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-04-25T01:47:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T01:47:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T01:47:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-04-25T01:47:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-04-25T01:47:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T01:47:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-04-25T01:47:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-04-25T01:47:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-04-25T01:47:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-04-25T01:47:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T01:47:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-04-25T01:47:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-04-25T01:47:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T01:47:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T01:47:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T01:47:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T01:47:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-04-25T01:47:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-04-25T01:47:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T01:47:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-04-25T01:47:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-04-25T01:47:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-04-25T01:47:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-04-25T01:47:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T01:47:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-04-25T01:47:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-04-25T01:47:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-04-25T01:47:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T01:47:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-04-25T01:47:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-04-25T01:47:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-04-25T01:47:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T01:47:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-04-25T01:47:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-04-25T01:47:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T01:47:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T01:47:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-04-25T01:47:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-04-25T01:47:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T01:47:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-04-25T01:47:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T01:47:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-04-25T01:47:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T01:47:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-04-25T01:47:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-04-25T01:47:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-04-25T01:47:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-04-25T01:47:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-04-25T01:47:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T01:47:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-04-25T01:47:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-04-25T01:47:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T01:47:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-04-25T01:47:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-04-25T01:47:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.605 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.606 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.606 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.606 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.606 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.606 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.606 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T01:47:26.606 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.606 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.606 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.606 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-04-25T01:47:26.606 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-04-25T01:47:26.606 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.607 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.607 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.607 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T01:47:26.607 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.607 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.607 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.607 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T01:47:26.607 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.607 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.607 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.607 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-04-25T01:47:26.607 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-04-25T01:47:26.607 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.608 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.608 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.608 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T01:47:26.608 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.608 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.608 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.608 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-04-25T01:47:26.608 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.608 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.608 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.608 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-04-25T01:47:26.608 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-04-25T01:47:26.608 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.609 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.609 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.609 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T01:47:26.609 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.609 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.609 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.609 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-04-25T01:47:26.609 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.609 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.609 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.609 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-04-25T01:47:26.609 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-04-25T01:47:26.609 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.610 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.610 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.610 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T01:47:26.610 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.610 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.610 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.610 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T01:47:26.610 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.610 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.610 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.610 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-04-25T01:47:26.610 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-04-25T01:47:26.611 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.611 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.611 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.611 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T01:47:26.611 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.611 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.611 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.611 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-04-25T01:47:26.611 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.611 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.611 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.611 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-04-25T01:47:26.612 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-04-25T01:47:26.612 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.612 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.612 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.612 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T01:47:26.612 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.612 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.612 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.612 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-04-25T01:47:26.612 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.612 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.612 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.612 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-04-25T01:47:26.613 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-04-25T01:47:26.613 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.613 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.613 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.613 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T01:47:26.613 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.613 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.613 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.613 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-04-25T01:47:26.613 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.613 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.613 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.613 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-04-25T01:47:26.614 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-04-25T01:47:26.614 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.614 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.614 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.614 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T01:47:26.614 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.614 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.614 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.614 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-04-25T01:47:26.614 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.614 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.614 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.615 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-04-25T01:47:26.615 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-04-25T01:47:26.615 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.615 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.615 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.615 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T01:47:26.615 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.615 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.615 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.615 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-04-25T01:47:26.615 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.615 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.615 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.615 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.616 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-04-25T01:47:26.616 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-04-25T01:47:26.616 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.616 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.616 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.616 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T01:47:26.616 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.616 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.616 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.616 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-04-25T01:47:26.616 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.616 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.616 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.617 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-04-25T01:47:26.617 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-04-25T01:47:26.617 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.617 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.617 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.617 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T01:47:26.617 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.617 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.617 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.617 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T01:47:26.617 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.617 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.617 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.618 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-04-25T01:47:26.618 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-04-25T01:47:26.618 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.618 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.618 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.618 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T01:47:26.618 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.618 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.618 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.618 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-04-25T01:47:26.618 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.618 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.619 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.619 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-04-25T01:47:26.619 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-04-25T01:47:26.619 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.619 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.619 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.619 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T01:47:26.619 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.619 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.619 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.620 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-04-25T01:47:26.621 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-04-25T01:47:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-04-25T01:47:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T01:47:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-04-25T01:47:26.622 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.623 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.623 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.623 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-04-25T01:47:26.623 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-04-25T01:47:26.623 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.623 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.623 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.623 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.624 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.624 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.624 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.624 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T01:47:26.624 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.624 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.624 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.624 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-04-25T01:47:26.624 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-04-25T01:47:26.624 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.625 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.625 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.625 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:26.625 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.625 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.625 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.625 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T01:47:26.625 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.625 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.625 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.625 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-04-25T01:47:26.625 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T01:47:26.626 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.626 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.626 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.626 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-04-25T01:47:26.626 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.626 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.626 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.626 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T01:47:26.626 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.626 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.626 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.626 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.626 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-04-25T01:47:26.627 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-04-25T01:47:26.627 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.627 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.627 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.627 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-04-25T01:47:26.627 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.627 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.627 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.627 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T01:47:26.627 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.627 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.627 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.627 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.628 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-04-25T01:47:26.628 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-04-25T01:47:26.628 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.628 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.628 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.628 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-04-25T01:47:26.628 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.628 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.628 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.628 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-04-25T01:47:26.628 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.628 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.629 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.629 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-04-25T01:47:26.629 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-04-25T01:47:26.629 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.629 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.629 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.629 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.629 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.629 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.629 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.629 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T01:47:26.629 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.629 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.630 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.630 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-04-25T01:47:26.630 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-04-25T01:47:26.630 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.630 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.630 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.630 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:26.630 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.631 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.631 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.631 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-04-25T01:47:26.631 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.631 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.631 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.632 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-04-25T01:47:26.632 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-04-25T01:47:26.632 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.632 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.632 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.632 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T01:47:26.632 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.632 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.632 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.632 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T01:47:26.632 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.632 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.632 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.633 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-04-25T01:47:26.633 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-04-25T01:47:26.633 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.633 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.633 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.633 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:26.633 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.633 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.633 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.633 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-04-25T01:47:26.633 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.633 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.633 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.634 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-04-25T01:47:26.634 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-04-25T01:47:26.634 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.634 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.634 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.634 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.634 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.634 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.634 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.634 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T01:47:26.634 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.634 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.634 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.635 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-04-25T01:47:26.635 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-04-25T01:47:26.635 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.635 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.635 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.635 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.635 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.635 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.635 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.635 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T01:47:26.635 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.635 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.635 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.636 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.636 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.636 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.636 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.636 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.636 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.636 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.636 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.636 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.636 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.636 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.636 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.636 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.637 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.637 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.637 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.637 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.637 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.637 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.637 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.637 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-04-25T01:47:26.637 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-04-25T01:47:26.637 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.637 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.638 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.638 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.638 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.638 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.638 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.638 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T01:47:26.638 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.638 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.638 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.638 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.639 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.639 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.639 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.639 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.639 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.639 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.639 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.639 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.639 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T01:47:26.640 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.640 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.640 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.640 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.640 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.640 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.640 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.640 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.641 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.641 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.641 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.641 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.641 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.641 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.641 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.641 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.641 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.641 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.641 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.641 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.641 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.641 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.642 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.642 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.642 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.642 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.642 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.642 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.642 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.642 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.642 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.642 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.643 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.643 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.643 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.643 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.643 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T01:47:26.643 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T01:47:26.643 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.643 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.644 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.644 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.644 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.644 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.644 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.644 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T01:47:26.644 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.645 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.645 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.645 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T01:47:26.645 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T01:47:26.645 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.645 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.645 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.645 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-04-25T01:47:26.645 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.645 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.645 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.645 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T01:47:26.646 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.646 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.646 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.646 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-04-25T01:47:26.646 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-04-25T01:47:26.646 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.646 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.646 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.646 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:26.646 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.646 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.646 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.647 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-04-25T01:47:26.647 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.647 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.647 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.647 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-04-25T01:47:26.647 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-04-25T01:47:26.647 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.647 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.647 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.647 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:26.647 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.647 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.648 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.649 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-04-25T01:47:26.649 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.649 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.649 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.649 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-04-25T01:47:26.649 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-04-25T01:47:26.649 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.649 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.649 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.649 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-04-25T01:47:26.649 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.649 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.650 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.650 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-04-25T01:47:26.650 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-04-25T01:47:26.650 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.650 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T01:47:26.650 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.650 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.650 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.650 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-04-25T01:47:26.650 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-04-25T01:47:26.650 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.650 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.651 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.651 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-04-25T01:47:26.651 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.651 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.651 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.651 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T01:47:26.651 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.651 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.651 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.651 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-04-25T01:47:26.651 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-04-25T01:47:26.651 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.651 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.652 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.652 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T01:47:26.652 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.652 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.652 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.652 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T01:47:26.652 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.652 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.652 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.652 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-04-25T01:47:26.652 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-04-25T01:47:26.653 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.653 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.653 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.653 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.653 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.654 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.654 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.654 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T01:47:26.654 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.654 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.654 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.655 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-04-25T01:47:26.655 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-04-25T01:47:26.655 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.655 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.655 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.655 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.656 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.656 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.656 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.656 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T01:47:26.656 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.656 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.656 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.656 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-04-25T01:47:26.656 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-04-25T01:47:26.657 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.657 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.657 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.657 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.657 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.657 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.657 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.657 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T01:47:26.657 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.657 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.657 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.658 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-04-25T01:47:26.658 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-04-25T01:47:26.658 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.658 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.658 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.658 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.658 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.658 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.658 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.658 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T01:47:26.658 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.658 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.659 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.659 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-04-25T01:47:26.659 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-04-25T01:47:26.659 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.659 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.659 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.659 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.659 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.659 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.659 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.660 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T01:47:26.660 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.660 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.660 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.660 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-04-25T01:47:26.660 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-04-25T01:47:26.660 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.660 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.661 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.661 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-04-25T01:47:26.661 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.661 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.661 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.661 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-04-25T01:47:26.661 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.661 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.661 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.661 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-04-25T01:47:26.661 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-04-25T01:47:26.662 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.662 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.662 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.662 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.662 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.662 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.662 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.662 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T01:47:26.662 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.662 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.663 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.663 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-04-25T01:47:26.663 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-04-25T01:47:26.663 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.663 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.663 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.663 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.663 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.663 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.664 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.664 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T01:47:26.664 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.664 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.664 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.664 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-04-25T01:47:26.664 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-04-25T01:47:26.664 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.664 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.664 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.665 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.665 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.665 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.665 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.665 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T01:47:26.665 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.665 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.665 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.665 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-04-25T01:47:26.665 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-04-25T01:47:26.666 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.666 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.666 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.666 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.666 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.666 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.666 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.667 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T01:47:26.667 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.667 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.667 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.667 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-04-25T01:47:26.667 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-04-25T01:47:26.667 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.667 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.667 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.667 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.667 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.667 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.667 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.668 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T01:47:26.668 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.668 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.668 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.668 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-04-25T01:47:26.668 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-04-25T01:47:26.668 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.668 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.668 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.668 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.669 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.669 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.669 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.669 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T01:47:26.669 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.669 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.669 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.669 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.669 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.670 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-04-25T01:47:26.670 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.670 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.670 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.670 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.670 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.670 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.670 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.670 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.671 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.671 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.671 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.671 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.671 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.671 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.671 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.671 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.671 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.671 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.671 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.671 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.671 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.672 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.672 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.672 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.672 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.672 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.672 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.672 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.672 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.672 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.672 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.672 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.672 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.672 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.673 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.673 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.673 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.673 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.673 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.673 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.673 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.673 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.673 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.673 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.673 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.673 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.673 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.674 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.674 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.674 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.674 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.674 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.674 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.674 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.674 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.674 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.674 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.674 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.674 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.675 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.675 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.675 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.675 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.675 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.675 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.675 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.675 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.675 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.675 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.675 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-04-25T01:47:26.675 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-04-25T01:47:26.676 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.676 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.676 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.676 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-04-25T01:47:26.676 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.676 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.676 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.676 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T01:47:26.676 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.676 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.676 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.676 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-04-25T01:47:26.676 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-04-25T01:47:26.677 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.677 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.677 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.677 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-04-25T01:47:26.677 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.677 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.677 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.677 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T01:47:26.677 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.677 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.677 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.677 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.677 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.678 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-04-25T01:47:26.678 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.678 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.678 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.678 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T01:47:26.678 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T01:47:26.678 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.678 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.678 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.678 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:26.678 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.679 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.679 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.679 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.679 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.679 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.679 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.679 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T01:47:26.679 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T01:47:26.679 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.679 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.679 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.679 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.680 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.680 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.680 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.680 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.680 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.680 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.680 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.680 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T01:47:26.680 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T01:47:26.680 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.680 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.680 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.680 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.681 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.681 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.681 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.681 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.681 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.681 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.681 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.681 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-04-25T01:47:26.681 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T01:47:26.681 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.681 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.681 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T01:47:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T01:47:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.682 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.683 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.683 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.683 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.683 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.683 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.683 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.683 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.683 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.683 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T01:47:26.683 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T01:47:26.683 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.683 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.683 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.684 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.684 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.684 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.684 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.684 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.684 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.684 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.684 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.684 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T01:47:26.684 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T01:47:26.684 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.685 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.685 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.685 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.685 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.685 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.685 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.685 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.685 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.685 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.685 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.685 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T01:47:26.686 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T01:47:26.686 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.686 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.686 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.686 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.686 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.686 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.686 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.686 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.686 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.687 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.687 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.687 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T01:47:26.687 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T01:47:26.687 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.687 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.687 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.687 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.688 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.688 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.688 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.688 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.688 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.688 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.688 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.688 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T01:47:26.688 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T01:47:26.688 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.688 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.688 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.688 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.689 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.689 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.689 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.689 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.689 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.689 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.689 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.689 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T01:47:26.689 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T01:47:26.689 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.689 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.690 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.690 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.690 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.690 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.690 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.690 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.691 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.691 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.691 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T01:47:26.691 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T01:47:26.692 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.692 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.692 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.692 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.692 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.692 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.693 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.693 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.693 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.693 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.693 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.693 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T01:47:26.694 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T01:47:26.694 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.694 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.694 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.694 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.695 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.695 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.695 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.695 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.695 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.695 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.695 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.696 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T01:47:26.696 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T01:47:26.696 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.696 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.696 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.697 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.697 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.697 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.697 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.697 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.697 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.698 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.698 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.698 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T01:47:26.698 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T01:47:26.698 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.699 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.699 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.699 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T01:47:26.699 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.699 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.699 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.700 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.700 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.700 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.700 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.700 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T01:47:26.700 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T01:47:26.701 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.701 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.701 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.701 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.701 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.701 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.701 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.702 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.702 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.702 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.702 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.702 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-04-25T01:47:26.702 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-04-25T01:47:26.702 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.702 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.702 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.702 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.703 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.703 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.703 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.703 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.703 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.703 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.703 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.703 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T01:47:26.703 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T01:47:26.704 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.704 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.704 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.704 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T01:47:26.704 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.704 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.704 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.704 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.704 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.705 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.705 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.705 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T01:47:26.705 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T01:47:26.705 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.705 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.705 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.705 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.705 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.705 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.705 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.705 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.706 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.706 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.706 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.706 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T01:47:26.706 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T01:47:26.706 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.706 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.706 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.707 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.707 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.707 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.707 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.707 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.707 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.707 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.707 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.707 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T01:47:26.707 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T01:47:26.708 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.708 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.708 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.708 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.708 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.708 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.708 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.708 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.708 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.708 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.708 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.708 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T01:47:26.709 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T01:47:26.709 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.709 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.709 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.709 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.709 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.709 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.709 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.709 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.709 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.709 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.709 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.709 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T01:47:26.710 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T01:47:26.710 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.710 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.710 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.710 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.710 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.710 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.710 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.710 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.710 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.710 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.710 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.711 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T01:47:26.711 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T01:47:26.711 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.711 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.711 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.711 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.711 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.711 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.711 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.711 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.711 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.711 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.711 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.712 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T01:47:26.712 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T01:47:26.712 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.712 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.712 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.712 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.712 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.712 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.712 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.712 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.712 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.712 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.712 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.713 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T01:47:26.713 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T01:47:26.713 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.713 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.713 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.713 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.713 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.713 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.713 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.713 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.713 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.713 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.713 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.714 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T01:47:26.714 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T01:47:26.714 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.714 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.714 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.714 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.714 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.714 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.714 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.714 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.714 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.714 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.714 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.715 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T01:47:26.715 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T01:47:26.715 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.715 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.715 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.715 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.715 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.715 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.715 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.715 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.715 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.715 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.715 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.716 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T01:47:26.716 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T01:47:26.716 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.716 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.716 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.716 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.716 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.716 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.716 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.716 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.717 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.717 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.717 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.717 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T01:47:26.717 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T01:47:26.717 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.717 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.717 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.717 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.717 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.718 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.718 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.718 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.718 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.718 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.718 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.718 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T01:47:26.718 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T01:47:26.718 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.718 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.718 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.718 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.718 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.719 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.720 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.720 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.720 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.720 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.720 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.720 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T01:47:26.720 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T01:47:26.720 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.720 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.720 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.720 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T01:47:26.720 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.721 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.721 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.721 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.721 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.721 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.721 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.721 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T01:47:26.721 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T01:47:26.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.721 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.721 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.721 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:26.721 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.722 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.722 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.722 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.722 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.722 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.722 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.722 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T01:47:26.722 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T01:47:26.722 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.722 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.722 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.722 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-04-25T01:47:26.723 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.723 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.723 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.723 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.723 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.723 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.723 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.723 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T01:47:26.723 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T01:47:26.723 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.723 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.723 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.723 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-04-25T01:47:26.724 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.724 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.724 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.724 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.724 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.724 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.724 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.724 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T01:47:26.724 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T01:47:26.724 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.724 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.724 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.724 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.725 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.725 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.725 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.725 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.725 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.725 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.725 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.725 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T01:47:26.725 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T01:47:26.725 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.725 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.725 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.725 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T01:47:26.726 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.726 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.726 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.726 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.726 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.726 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.726 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.726 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T01:47:26.726 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T01:47:26.726 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.726 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.726 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.727 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.728 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.728 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.728 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.728 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.728 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.728 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.728 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.728 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T01:47:26.728 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T01:47:26.728 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.728 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.728 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.728 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:26.729 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.730 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.730 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.730 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.730 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.730 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.730 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.730 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T01:47:26.730 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T01:47:26.730 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.730 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.730 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.730 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T01:47:26.730 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.731 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.732 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.732 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.732 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.732 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.732 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.732 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-04-25T01:47:26.732 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-04-25T01:47:26.732 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.732 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.732 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.732 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.732 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.732 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.733 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.733 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.733 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.733 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.733 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.733 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-04-25T01:47:26.733 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-04-25T01:47:26.733 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.733 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.733 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.733 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.733 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.734 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.734 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.734 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-04-25T01:47:26.734 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.734 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.734 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.734 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-04-25T01:47:26.734 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-04-25T01:47:26.734 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.734 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.734 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.734 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-04-25T01:47:26.734 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.735 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.735 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.735 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.735 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.735 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.735 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.735 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-04-25T01:47:26.735 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-04-25T01:47:26.735 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.735 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.735 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.735 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.735 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.736 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.736 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.736 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.736 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.736 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.736 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.736 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.736 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.736 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.736 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.736 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.736 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.736 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.737 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.737 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.737 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.737 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.737 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.737 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.737 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.737 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.737 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.737 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.737 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.737 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.737 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.738 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.738 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.738 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.738 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.738 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.738 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.738 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.738 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.738 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.738 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.738 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.738 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.738 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.739 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.739 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.739 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.739 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.739 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.739 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.739 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.739 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.739 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.739 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.739 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.739 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.739 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.739 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.740 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.741 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.741 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.741 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.741 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.741 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.741 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-04-25T01:47:26.741 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-04-25T01:47:26.741 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.741 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.741 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.741 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.741 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.741 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.742 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T01:47:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T01:47:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-04-25T01:47:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.743 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.744 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.745 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.746 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.746 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.746 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-04-25T01:47:26.746 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-04-25T01:47:26.746 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.746 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.746 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.746 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-04-25T01:47:26.746 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.746 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.746 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.747 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.747 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.747 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.747 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.747 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-04-25T01:47:26.747 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-04-25T01:47:26.747 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.747 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.747 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.747 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-04-25T01:47:26.747 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.747 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.748 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.748 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.748 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-04-25T01:47:26.748 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.748 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.748 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.748 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.748 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.748 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-04-25T01:47:26.748 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-04-25T01:47:26.748 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.748 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.748 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.749 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-04-25T01:47:26.749 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-04-25T01:47:26.749 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-04-25T01:47:26.749 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.749 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.749 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.749 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.749 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.749 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-04-25T01:47:26.749 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-04-25T01:47:26.749 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.749 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.749 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.750 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.750 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.750 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.750 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.750 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.750 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.750 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.750 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.750 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.750 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.750 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.750 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-04-25T01:47:26.750 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.751 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.751 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.751 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-04-25T01:47:26.751 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-04-25T01:47:26.751 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.751 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.751 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.751 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.751 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.751 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.751 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.751 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-04-25T01:47:26.751 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.751 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.752 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.752 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.752 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.752 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.752 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.752 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.752 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.752 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.752 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.752 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.752 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.752 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.752 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.753 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.753 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.753 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.753 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.753 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.753 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.753 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.753 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.753 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.753 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.753 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.753 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.753 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.754 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.754 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.754 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.754 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.754 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.754 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.754 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.754 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.754 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.754 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.754 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.755 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.755 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.755 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.755 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.755 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.755 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.755 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.755 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.755 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.755 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.755 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.755 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.755 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.755 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.756 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.756 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.756 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.756 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.756 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.756 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.756 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.756 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.756 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.756 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.756 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.756 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.756 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.757 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.757 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.757 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.757 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-04-25T01:47:26.757 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-04-25T01:47:26.757 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.757 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.757 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.757 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-04-25T01:47:26.757 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.757 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.757 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.757 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T01:47:26.758 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.758 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.758 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.758 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-04-25T01:47:26.758 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-04-25T01:47:26.758 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.758 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.758 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.758 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-04-25T01:47:26.758 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.758 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.758 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.758 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T01:47:26.759 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.759 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.759 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.759 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-04-25T01:47:26.759 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-04-25T01:47:26.759 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.759 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.759 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.759 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-04-25T01:47:26.759 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.759 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.759 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.759 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-04-25T01:47:26.760 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.760 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.760 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.760 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-04-25T01:47:26.760 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-04-25T01:47:26.760 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.760 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.760 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.760 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-04-25T01:47:26.760 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.760 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.760 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.760 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T01:47:26.761 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.762 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.762 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.762 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.762 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:26.762 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:26.762 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.762 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.762 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.762 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:26.762 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.762 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.762 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.762 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T01:47:26.762 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.763 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.763 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.763 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-04-25T01:47:26.763 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-04-25T01:47:26.763 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.763 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.763 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.763 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-04-25T01:47:26.763 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.763 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.763 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.763 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T01:47:26.763 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.764 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.764 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.764 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.764 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.764 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-04-25T01:47:26.764 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.764 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.764 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.764 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.764 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.764 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.765 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.766 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.766 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.766 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.766 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.766 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.766 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.766 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.766 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.766 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.766 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.766 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.766 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.767 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.767 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.767 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.767 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.767 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.767 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.767 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.767 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.767 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.767 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.767 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.767 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.767 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.768 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.768 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.768 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.768 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.768 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.768 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.768 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.768 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.768 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.768 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.768 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.768 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.768 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.769 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.770 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.770 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-04-25T01:47:26.770 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.770 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.770 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.770 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.770 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.770 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.770 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.770 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-04-25T01:47:26.770 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-04-25T01:47:26.770 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.771 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.772 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.772 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:26.772 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.772 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.772 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.772 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.772 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.772 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.772 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.772 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.772 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.772 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.772 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-04-25T01:47:26.773 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-04-25T01:47:26.773 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-04-25T01:47:26.773 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.773 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-04-25T01:47:26.773 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-04-25T01:47:26.773 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.773 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.773 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.773 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-04-25T01:47:26.773 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.773 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.773 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.773 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T01:47:26.773 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.774 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.774 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.774 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-04-25T01:47:26.774 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-04-25T01:47:26.774 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.774 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.774 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.774 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-04-25T01:47:26.774 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.774 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.774 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.774 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-04-25T01:47:26.775 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.776 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.776 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.776 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:26.776 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:26.776 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.776 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.776 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.776 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-04-25T01:47:26.776 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-04-25T01:47:26.776 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.776 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.776 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T01:47:26.776 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.777 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.778 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.778 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.778 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.778 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.778 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.778 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.778 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.778 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.778 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.778 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.778 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.778 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.778 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.779 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.779 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.779 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.779 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.779 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.779 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.779 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.779 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.779 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.779 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.779 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.779 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.779 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.780 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.781 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.782 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-04-25T01:47:26.782 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-04-25T01:47:26.782 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.782 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.782 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.782 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.782 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.782 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.782 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.782 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-04-25T01:47:26.782 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.782 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.783 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.785 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-04-25T01:47:26.785 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-04-25T01:47:26.785 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.785 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.785 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.785 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T01:47:26.785 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T01:47:26.785 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-04-25T01:47:26.785 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.785 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T01:47:26.785 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.785 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.785 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.786 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-04-25T01:47:26.786 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-04-25T01:47:26.786 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.786 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.786 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.786 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.786 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.786 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.786 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.786 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-04-25T01:47:26.786 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.786 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.786 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.787 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-04-25T01:47:26.787 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-04-25T01:47:26.787 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.787 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.787 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.787 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-04-25T01:47:26.787 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.787 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.787 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.787 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T01:47:26.787 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.787 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.787 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.788 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.788 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.788 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-04-25T01:47:26.788 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.788 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.788 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.788 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.788 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.788 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.788 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.789 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.789 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.789 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.789 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.789 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.789 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.789 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.789 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.789 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.789 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.789 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.789 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.789 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.789 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.790 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.790 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.790 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.790 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.790 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.790 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.790 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.790 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.790 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.790 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.790 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.790 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.791 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.791 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.791 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.791 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.791 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.791 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.791 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.791 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.791 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.791 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.791 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.791 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-04-25T01:47:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.796 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.796 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.796 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.796 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.796 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.796 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.796 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.796 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.796 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.796 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.796 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.796 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.796 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.797 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.797 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.797 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.797 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.797 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.797 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.797 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.797 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.797 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.797 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.797 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.797 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.797 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.798 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.798 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.798 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.798 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.798 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.798 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.798 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.798 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.798 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.798 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.798 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.798 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.799 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.799 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.799 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.799 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.799 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.799 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.799 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.799 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.799 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.799 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.799 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-04-25T01:47:26.799 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.800 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.800 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.800 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-04-25T01:47:26.800 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-04-25T01:47:26.800 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.800 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.800 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.800 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-04-25T01:47:26.800 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.800 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.800 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.800 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T01:47:26.800 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.800 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.801 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.801 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-04-25T01:47:26.801 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-04-25T01:47:26.801 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.801 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.801 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.801 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:26.801 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-04-25T01:47:26.801 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.801 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.801 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T01:47:26.802 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.802 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.802 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.802 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.802 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.802 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.802 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.802 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.802 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.802 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.802 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.802 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.802 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.803 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.803 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.803 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.803 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.803 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.803 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.803 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.803 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.803 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.803 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.803 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.803 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.803 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.804 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.804 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.804 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.804 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.804 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.804 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.804 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.804 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.804 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.804 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.804 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.804 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.804 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.805 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.805 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.805 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.805 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.805 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.805 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.805 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.805 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.805 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.805 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.805 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.805 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.805 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.805 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.806 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.806 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.806 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.806 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.806 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.806 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.806 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.806 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.806 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.806 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.806 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.806 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.806 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.807 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.807 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.807 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.809 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.809 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.809 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.810 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-04-25T01:47:26.810 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.810 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.810 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.810 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-04-25T01:47:26.810 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-04-25T01:47:26.810 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.810 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.810 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.810 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-04-25T01:47:26.810 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.810 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.810 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.811 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.812 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.812 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.812 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.812 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.812 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.812 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.812 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.812 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.812 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.812 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.812 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.812 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.812 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.813 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.813 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.813 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.813 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.813 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.813 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.813 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.813 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.813 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.813 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.813 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.813 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.813 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.814 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.814 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.814 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.814 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.814 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.814 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.814 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.814 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.814 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.814 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.814 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.814 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.814 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.814 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.815 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T01:47:26.816 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-04-25T01:47:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-04-25T01:47:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-04-25T01:47:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-04-25T01:47:26.818 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.818 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.818 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.818 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-04-25T01:47:26.818 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-04-25T01:47:26.818 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.818 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.818 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.818 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-04-25T01:47:26.818 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.818 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.818 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.819 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-04-25T01:47:26.820 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.820 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.820 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.820 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-04-25T01:47:26.820 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-04-25T01:47:26.820 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.820 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.820 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.820 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-04-25T01:47:26.820 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.820 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.820 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.820 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T01:47:26.821 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T01:47:26.821 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.821 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.821 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.821 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.821 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.821 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-04-25T01:47:26.821 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.821 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.821 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.821 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.821 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.821 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.822 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.823 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.824 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.825 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-04-25T01:47:26.826 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.827 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.828 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.829 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.830 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.831 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-04-25T01:47:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.832 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.833 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.834 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.835 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.835 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.835 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.835 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.835 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.835 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.835 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.835 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.835 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.835 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.835 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.835 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.835 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.836 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.836 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.836 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.836 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.836 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.836 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.836 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.836 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.836 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.836 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.836 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.836 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.836 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.837 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.838 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.838 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-04-25T01:47:26.838 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-04-25T01:47:26.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.838 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.838 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.838 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.838 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.838 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.838 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.838 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T01:47:26.838 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.838 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.838 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.839 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.839 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.839 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.839 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.839 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.839 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.839 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.839 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.839 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.839 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.839 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.839 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.839 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.840 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.841 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.841 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.841 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.841 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.841 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.841 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.841 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.841 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.841 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.841 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.841 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.841 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.841 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.842 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.842 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.842 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.842 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.842 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.842 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.842 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.842 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.842 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.842 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.842 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.842 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.842 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.844 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.845 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.845 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.845 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.845 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-04-25T01:47:26.845 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.845 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.845 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.845 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.845 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.845 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.846 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.846 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.846 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.846 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.846 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.846 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.846 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.846 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.846 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.846 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.846 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.846 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.846 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.846 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.848 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.848 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.848 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.848 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.848 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.848 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.848 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.848 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.848 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.848 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.848 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.848 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.848 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.849 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.850 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.850 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.850 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.850 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.850 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.850 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.850 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.850 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.850 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.850 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.850 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.850 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.850 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-04-25T01:47:26.851 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.851 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.851 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.851 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.851 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.851 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.851 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.851 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.851 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.851 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.851 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.851 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.851 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.851 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.852 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.853 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.854 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.854 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.854 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.854 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.854 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.854 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.854 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.854 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.854 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.854 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.854 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.854 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.854 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.854 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.855 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-04-25T01:47:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.856 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.857 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.858 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.858 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.858 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.858 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.858 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.858 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.858 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.858 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.858 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.858 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.858 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.858 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.858 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.859 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.859 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.859 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.859 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.859 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.859 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.859 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.859 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.859 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.859 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.859 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.859 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.859 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.860 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.860 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.860 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.860 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.860 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.860 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.860 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.860 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.860 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.860 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.860 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.860 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.860 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.861 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.861 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.861 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.861 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.861 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.861 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.861 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:26.861 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:26.861 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.861 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.861 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.861 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:26.861 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.862 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.862 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.862 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.862 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.862 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.862 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.862 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-04-25T01:47:26.862 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-04-25T01:47:26.862 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:26.862 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.862 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.862 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.863 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.864 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-04-25T01:47:26.864 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-04-25T01:47:26.864 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.864 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.864 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.864 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.864 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.864 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.864 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.864 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T01:47:26.864 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.864 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.864 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.865 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-04-25T01:47:26.865 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-04-25T01:47:26.865 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.865 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.865 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.865 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.866 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.866 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.866 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.866 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.866 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.866 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.866 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.866 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.866 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.866 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.866 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.866 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.867 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.867 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.867 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.867 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.867 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.867 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.867 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.867 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.867 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.867 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.867 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.867 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.867 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.867 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.868 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.869 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.870 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.870 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.870 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.870 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.870 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.870 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.870 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.870 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.870 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.870 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.870 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.870 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-04-25T01:47:26.871 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.872 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.873 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.873 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-04-25T01:47:26.873 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.873 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.873 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.873 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.873 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.873 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-04-25T01:47:26.873 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.873 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.873 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.874 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-04-25T01:47:26.874 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-04-25T01:47:26.874 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.874 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.874 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.874 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.874 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.874 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.874 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.874 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.874 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.874 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.874 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.876 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.876 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.876 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.876 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.876 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.876 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.876 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.876 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.876 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.876 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.876 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.876 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.876 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.876 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.878 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.878 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.878 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.878 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.878 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.878 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.878 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.878 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.878 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.878 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.878 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.878 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.878 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.879 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.880 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.880 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.880 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.880 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.880 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.880 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.880 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.880 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.880 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.880 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.880 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.880 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.880 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T01:47:26.881 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T01:47:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-04-25T01:47:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-04-25T01:47:26.882 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T01:47:26.883 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.884 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-04-25T01:47:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T01:47:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-04-25T01:47:26.885 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-04-25T01:47:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-04-25T01:47:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-04-25T01:47:26.886 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-04-25T01:47:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-04-25T01:47:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-04-25T01:47:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-04-25T01:47:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.887 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-04-25T01:47:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-04-25T01:47:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-04-25T01:47:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-04-25T01:47:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-04-25T01:47:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.888 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.889 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.890 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.890 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.890 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.890 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.890 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.890 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.890 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.890 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.890 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.890 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.890 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.890 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.890 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.892 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.892 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.892 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.892 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.892 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.892 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.892 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.892 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.892 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.892 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.892 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.892 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.892 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.892 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.893 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.894 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.894 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.894 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.894 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.894 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.894 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-04-25T01:47:26.894 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-04-25T01:47:26.894 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.894 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.894 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.894 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:26.894 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.894 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.895 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.895 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.895 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.895 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.895 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.895 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-04-25T01:47:26.895 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T01:47:26.895 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.895 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.895 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.895 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.895 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.895 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.896 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-04-25T01:47:26.897 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.898 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.898 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.898 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.898 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.898 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.898 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.898 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.898 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.898 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.898 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.898 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-04-25T01:47:26.898 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-04-25T01:47:26.898 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.899 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.899 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.899 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.899 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.899 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.899 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.899 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.899 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.899 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.899 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.900 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.900 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.900 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.900 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.900 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.900 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.900 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.900 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.900 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.900 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.900 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.900 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.900 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.900 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.901 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.901 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.901 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.901 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.901 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.901 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.901 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.901 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.901 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.901 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.901 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.901 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.901 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.902 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.902 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.902 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.902 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.902 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.902 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.902 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.902 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.902 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.902 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.902 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.902 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.902 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.903 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.903 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.903 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.903 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.903 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.903 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.903 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.903 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.903 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.903 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.903 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.903 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.903 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T01:47:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T01:47:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.906 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T01:47:26.906 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T01:47:26.906 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.906 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.906 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.906 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.906 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.906 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.906 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.906 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.906 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.906 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.906 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.907 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.908 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.909 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.910 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.911 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.911 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.911 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.911 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.911 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.911 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.911 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.911 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.911 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.911 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.911 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.911 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.912 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.912 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.912 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.912 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.912 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.912 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.912 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.912 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-04-25T01:47:26.912 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.912 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.912 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.912 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.913 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.913 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.913 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.913 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.913 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.913 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.913 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.913 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.913 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.913 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.913 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.913 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.913 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.914 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.914 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.914 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.914 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.914 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.914 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.914 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.914 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.914 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.914 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.914 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.914 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.914 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.915 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.915 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.915 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.915 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.915 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.915 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.915 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.915 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.915 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.915 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.915 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.915 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.915 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.916 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.916 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.916 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.916 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.916 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.916 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.916 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.916 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.916 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.916 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.916 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.916 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.916 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.917 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.917 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.917 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.917 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.917 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.917 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.917 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.917 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.917 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.917 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.917 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.917 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.917 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.918 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-04-25T01:47:26.918 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-04-25T01:47:26.918 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.918 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.918 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.918 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-04-25T01:47:26.918 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.918 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.918 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.918 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T01:47:26.918 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.918 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.918 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.919 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.919 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.919 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-04-25T01:47:26.919 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.919 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.919 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.919 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.919 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.919 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.919 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.919 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.919 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.920 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.920 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.920 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.920 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.920 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.920 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.920 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.920 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.920 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.920 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.920 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.920 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.920 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.920 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.922 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.922 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.922 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.922 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.922 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.922 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.922 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.922 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.922 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.922 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.922 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.922 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.922 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.922 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.923 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.923 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.923 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.923 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.923 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.923 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.923 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.923 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.923 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.923 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.923 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.923 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.923 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-04-25T01:47:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-04-25T01:47:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.925 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.926 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.926 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.926 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.926 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.926 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.926 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-04-25T01:47:26.926 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-04-25T01:47:26.926 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.926 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.926 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.926 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.926 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.927 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.927 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.927 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.927 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.927 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.927 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.927 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-04-25T01:47:26.927 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-04-25T01:47:26.927 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.927 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.927 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.927 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.928 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.928 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.928 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.928 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.928 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.928 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.928 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.928 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-04-25T01:47:26.928 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-04-25T01:47:26.928 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:26.928 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.928 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.928 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.929 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.929 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.929 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.929 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.929 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.929 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.929 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.929 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-04-25T01:47:26.929 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-04-25T01:47:26.929 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.929 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.929 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.929 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.930 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.930 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.930 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.930 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.930 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.930 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.930 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.930 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-04-25T01:47:26.930 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-04-25T01:47:26.930 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.930 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.930 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.930 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.931 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.931 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.931 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.931 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.931 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.931 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.931 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.931 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-04-25T01:47:26.931 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-04-25T01:47:26.931 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.931 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.931 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.931 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.932 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.932 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.932 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.932 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.932 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.932 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.932 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.932 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-04-25T01:47:26.932 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-04-25T01:47:26.932 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.932 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.932 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.933 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.933 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T01:47:26.933 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-04-25T01:47:26.933 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.933 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.933 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.933 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.933 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.933 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-04-25T01:47:26.933 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-04-25T01:47:26.933 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.933 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.933 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.934 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.934 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.934 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.934 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.934 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.934 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.934 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.934 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.934 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-04-25T01:47:26.934 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-04-25T01:47:26.934 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.934 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.935 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.935 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.935 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.935 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.935 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.935 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.935 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.935 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.935 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-04-25T01:47:26.935 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-04-25T01:47:26.935 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.935 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-04-25T01:47:26.936 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.937 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.937 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.937 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-04-25T01:47:26.937 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-04-25T01:47:26.937 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.937 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.937 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.937 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.937 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.937 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.937 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.937 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T01:47:26.937 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.937 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.938 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.938 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-04-25T01:47:26.938 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-04-25T01:47:26.938 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.938 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.938 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.938 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.938 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.938 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.938 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.938 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-04-25T01:47:26.938 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.938 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.939 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.940 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.940 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.940 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.940 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.940 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.940 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.940 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.940 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.940 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.940 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.940 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.940 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.940 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.941 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.941 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.941 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.941 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.941 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.941 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.941 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.941 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.941 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.941 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.941 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.941 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.942 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.942 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.942 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.942 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.942 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.942 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.942 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.942 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.942 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.942 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.942 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.942 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.942 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.942 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.943 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.943 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.943 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.943 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.943 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.943 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.943 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.943 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.943 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.943 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.943 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.943 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.943 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.944 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.944 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.944 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.944 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-04-25T01:47:26.944 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.944 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.944 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.944 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.944 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.944 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.944 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.944 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.945 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.945 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.945 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.945 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.945 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.945 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.945 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.945 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.945 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.945 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.945 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.945 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.945 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.945 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.946 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.946 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.946 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.946 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.946 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.946 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.946 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.946 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.946 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.946 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.946 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.946 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.946 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.947 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.948 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.949 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.950 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.951 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.951 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.951 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.951 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.951 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.951 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.951 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.951 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.951 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.951 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.951 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.951 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.951 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.952 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.952 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.952 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.952 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.952 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.952 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.952 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.952 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.952 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.952 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.952 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.952 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.952 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.953 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.953 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.953 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.953 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.953 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.953 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.953 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.953 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.953 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.953 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.953 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.953 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.953 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.953 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.954 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.954 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.954 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.954 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.954 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.954 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.954 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.954 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.954 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.954 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.954 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.954 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.955 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.955 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.955 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.955 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-04-25T01:47:26.955 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.955 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.955 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.955 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-04-25T01:47:26.955 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-04-25T01:47:26.955 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.955 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.955 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.956 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T01:47:26.956 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.956 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.956 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.956 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.956 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.956 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.956 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.956 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:26.956 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:26.956 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:26.956 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.957 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.957 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-04-25T01:47:26.957 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.957 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.957 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.957 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.957 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.957 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.957 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.957 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.957 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.957 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.957 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.958 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.958 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.958 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.958 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.958 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.958 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.958 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.958 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.958 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.958 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.958 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.958 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.958 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.958 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.959 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.959 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.959 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.959 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.959 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.959 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.959 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.959 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.959 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.959 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.959 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.959 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.960 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.961 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.961 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.961 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.961 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.961 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.961 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.961 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.961 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.961 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.961 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.961 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.961 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.961 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.962 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.962 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.962 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.962 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.962 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.962 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.962 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.962 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.962 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.962 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.962 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.962 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-04-25T01:47:26.963 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.963 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.963 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.963 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-04-25T01:47:26.963 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-04-25T01:47:26.963 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.963 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.963 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.963 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.963 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.963 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.963 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.963 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T01:47:26.964 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.964 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.964 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.964 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-04-25T01:47:26.964 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-04-25T01:47:26.964 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.964 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.964 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.964 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.964 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.964 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.964 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.964 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T01:47:26.965 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.965 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.965 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.965 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-04-25T01:47:26.965 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-04-25T01:47:26.965 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.965 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.965 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.965 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.965 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.965 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.965 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.965 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T01:47:26.966 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.966 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.966 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.966 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-04-25T01:47:26.966 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-04-25T01:47:26.966 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.966 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.966 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.966 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.966 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.966 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.966 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.966 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T01:47:26.967 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.968 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.968 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.968 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-04-25T01:47:26.968 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-04-25T01:47:26.968 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.968 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.968 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.968 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.968 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.968 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.968 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.968 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.968 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.969 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.969 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.969 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-04-25T01:47:26.969 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-04-25T01:47:26.969 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.969 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.969 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.969 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.969 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.969 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.969 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.969 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.970 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.970 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.970 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.970 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-04-25T01:47:26.970 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-04-25T01:47:26.970 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.970 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.970 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.970 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T01:47:26.970 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.970 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.970 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-04-25T01:47:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-04-25T01:47:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-04-25T01:47:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-04-25T01:47:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-04-25T01:47:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-04-25T01:47:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-04-25T01:47:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-04-25T01:47:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-04-25T01:47:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-04-25T01:47:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-04-25T01:47:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-04-25T01:47:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-04-25T01:47:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-04-25T01:47:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-04-25T01:47:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T01:47:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-04-25T01:47:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-04-25T01:47:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-04-25T01:47:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-04-25T01:47:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-04-25T01:47:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T01:47:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-04-25T01:47:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-04-25T01:47:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-04-25T01:47:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-04-25T01:47:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-04-25T01:47:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T01:47:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-04-25T01:47:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T01:47:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T01:47:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-04-25T01:47:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-04-25T01:47:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-04-25T01:47:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-04-25T01:47:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-04-25T01:47:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-04-25T01:47:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-04-25T01:47:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-04-25T01:47:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T01:47:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T01:47:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 24119, 2024-04-25T01:47:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "y", 2024-04-25T01:47:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr_features": 4540701547738038271, 2024-04-25T01:47:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-04-25T01:47:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-04-25T01:47:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-04-25T01:47:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-04-25T01:47:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-04-25T01:47:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-04-25T01:47:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-04-25T01:47:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-04-25T01:47:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-04-25T01:47:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-04-25T01:47:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-04-25T01:47:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-04-25T01:47:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-04-25T01:47:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-04-25T01:47:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T01:47:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-04-25T01:47:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-04-25T01:47:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-04-25T01:47:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-04-25T01:47:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-04-25T01:47:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-04-25T01:47:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T01:47:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-04-25T01:47:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-04-25T01:47:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-04-25T01:47:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T01:47:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-04-25T01:47:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T01:47:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-04-25T01:47:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-04-25T01:47:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T01:47:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T01:47:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T01:47:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-04-25T01:47:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-04-25T01:47:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-04-25T01:47:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-04-25T01:47:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-04-25T01:47:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-04-25T01:47:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T01:47:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-04-25T01:47:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-04-25T01:47:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-04-25T01:47:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T01:47:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T01:47:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-04-25T01:47:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-04-25T01:47:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-04-25T01:47:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-04-25T01:47:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T01:47:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T01:47:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-04-25T01:47:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-04-25T01:47:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T01:47:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-04-25T01:47:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-04-25T01:47:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-04-25T01:47:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-04-25T01:47:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-04-25T01:47:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-04-25T01:47:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-04-25T01:47:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-04-25T01:47:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-04-25T01:47:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-04-25T01:47:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-04-25T01:47:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T01:47:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-04-25T01:47:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T01:47:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-04-25T01:47:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-04-25T01:47:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-04-25T01:47:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T01:47:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-04-25T01:47:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-04-25T01:47:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T01:47:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T01:47:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-04-25T01:47:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-04-25T01:47:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T01:47:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-04-25T01:47:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T01:47:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-04-25T01:47:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T01:47:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-04-25T01:47:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-04-25T01:47:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-04-25T01:47:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-04-25T01:47:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-04-25T01:47:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T01:47:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-04-25T01:47:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-04-25T01:47:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T01:47:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-04-25T01:47:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-04-25T01:47:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T01:47:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-04-25T01:47:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-04-25T01:47:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T01:47:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-04-25T01:47:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-04-25T01:47:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T01:47:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T01:47:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-04-25T01:47:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-04-25T01:47:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T01:47:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-04-25T01:47:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-04-25T01:47:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-04-25T01:47:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T01:47:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T01:47:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-04-25T01:47:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-04-25T01:47:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T01:47:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-04-25T01:47:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-04-25T01:47:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-04-25T01:47:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T01:47:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-04-25T01:47:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-04-25T01:47:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-04-25T01:47:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T01:47:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-04-25T01:47:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-04-25T01:47:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-04-25T01:47:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T01:47:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-04-25T01:47:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-04-25T01:47:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-04-25T01:47:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T01:47:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T01:47:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-04-25T01:47:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-04-25T01:47:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T01:47:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-04-25T01:47:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-04-25T01:47:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-04-25T01:47:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T01:47:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T01:47:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-04-25T01:47:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-04-25T01:47:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T01:47:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-04-25T01:47:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-04-25T01:47:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-04-25T01:47:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T01:47:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-04-25T01:47:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-04-25T01:47:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-04-25T01:47:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T01:47:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-04-25T01:47:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T01:47:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-04-25T01:47:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T01:47:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-04-25T01:47:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-04-25T01:47:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-04-25T01:47:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T01:47:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-04-25T01:47:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-04-25T01:47:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-04-25T01:47:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-04-25T01:47:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-04-25T01:47:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-04-25T01:47:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-04-25T01:47:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-04-25T01:47:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-04-25T01:47:27.079 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.079 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.079 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.079 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T01:47:27.079 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.079 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.079 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.079 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T01:47:27.079 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.079 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.079 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.079 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-04-25T01:47:27.079 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-04-25T01:47:27.080 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.081 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.081 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.081 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.081 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.081 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.081 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.081 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T01:47:27.081 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.081 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.081 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.081 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-04-25T01:47:27.081 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-04-25T01:47:27.081 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.082 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.083 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.083 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.083 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.083 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.083 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.083 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.083 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.083 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.083 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.083 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.083 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.083 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.083 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.083 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.084 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.085 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.085 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.085 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.085 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.085 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.085 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.085 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.085 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.085 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.085 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.085 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T01:47:27.085 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.086 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.086 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.086 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.086 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.086 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.086 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.086 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.086 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.086 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.086 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.086 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.086 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.086 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.087 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.087 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.087 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.087 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.087 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.087 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.087 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.087 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.087 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.087 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.087 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.087 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.087 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.088 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.089 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.089 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.089 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.089 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.089 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T01:47:27.089 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.089 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.089 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.089 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T01:47:27.089 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T01:47:27.089 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.089 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.089 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.090 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-04-25T01:47:27.090 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.090 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.090 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.090 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T01:47:27.090 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.090 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.090 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.090 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-04-25T01:47:27.090 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-04-25T01:47:27.090 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.090 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.090 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.091 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.091 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.091 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.091 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.091 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-04-25T01:47:27.091 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.091 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.091 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.091 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-04-25T01:47:27.091 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-04-25T01:47:27.091 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.091 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.091 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.092 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.092 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.092 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.092 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.092 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T01:47:27.092 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.092 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.092 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.092 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-04-25T01:47:27.092 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-04-25T01:47:27.092 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.092 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.092 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.093 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-04-25T01:47:27.094 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.094 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.094 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.094 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-04-25T01:47:27.094 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-04-25T01:47:27.094 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.094 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T01:47:27.094 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.094 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.094 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.094 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-04-25T01:47:27.094 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-04-25T01:47:27.094 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.095 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.095 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.095 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-04-25T01:47:27.095 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.095 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.095 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.095 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T01:47:27.095 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.095 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.095 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.095 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-04-25T01:47:27.095 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-04-25T01:47:27.095 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.096 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.097 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.097 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.097 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.097 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.097 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.097 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T01:47:27.097 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.097 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.097 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.097 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-04-25T01:47:27.097 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-04-25T01:47:27.097 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.098 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.098 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.098 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.098 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.098 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.098 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.098 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T01:47:27.098 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.098 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.098 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.098 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-04-25T01:47:27.098 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-04-25T01:47:27.098 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.099 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.100 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.100 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.100 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.100 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.100 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.100 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T01:47:27.100 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.100 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.100 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.100 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-04-25T01:47:27.100 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-04-25T01:47:27.100 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.100 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.101 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.102 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-04-25T01:47:27.102 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.102 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.102 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.102 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-04-25T01:47:27.102 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.102 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.102 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.102 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-04-25T01:47:27.102 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-04-25T01:47:27.102 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.102 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.102 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.103 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.104 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.105 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.105 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.105 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T01:47:27.105 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.105 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.105 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.105 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-04-25T01:47:27.105 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-04-25T01:47:27.105 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.105 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.105 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.105 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.105 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T01:47:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-04-25T01:47:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-04-25T01:47:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.106 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.107 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.108 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.108 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T01:47:27.108 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.108 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.108 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.108 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.108 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.108 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-04-25T01:47:27.108 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.108 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.108 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.108 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.109 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.109 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.109 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.109 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.109 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.109 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.109 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.109 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.109 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.109 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.109 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.109 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.109 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.110 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.110 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.110 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.110 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.110 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.110 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.110 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.110 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.110 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.110 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.110 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.110 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.110 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.111 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.111 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.111 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.111 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.111 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.111 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.111 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.111 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.111 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.111 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.111 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.111 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.111 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.112 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.112 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.112 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.112 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.112 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.112 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.112 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.112 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.112 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.112 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.112 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.112 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.112 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.113 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.113 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.113 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.113 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.113 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.113 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.113 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.113 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.113 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.113 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.113 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.113 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.114 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.114 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-04-25T01:47:27.114 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-04-25T01:47:27.114 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.114 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.114 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.114 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-04-25T01:47:27.114 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.114 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.114 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.114 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T01:47:27.114 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.114 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.115 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.115 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-04-25T01:47:27.115 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-04-25T01:47:27.115 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.115 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.115 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.115 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-04-25T01:47:27.115 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.115 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.115 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.115 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T01:47:27.115 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.115 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.116 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.116 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.116 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.116 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-04-25T01:47:27.116 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.116 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.116 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.116 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T01:47:27.116 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T01:47:27.116 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.116 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.116 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.117 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.117 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.117 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.117 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.117 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.117 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.117 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.117 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.117 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T01:47:27.117 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T01:47:27.117 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.117 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.117 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.118 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.118 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.118 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.118 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.118 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.118 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.118 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.118 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.118 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T01:47:27.118 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T01:47:27.118 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.118 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.118 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.119 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.119 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.119 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.119 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.119 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.119 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.119 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.119 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.119 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-04-25T01:47:27.119 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T01:47:27.119 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.119 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.119 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.120 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.121 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.122 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.122 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.122 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.122 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.122 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.122 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.122 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T01:47:27.122 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T01:47:27.122 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.122 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.122 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.122 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.122 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.123 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.123 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.123 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.123 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.123 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.123 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.123 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T01:47:27.123 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T01:47:27.123 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.123 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.123 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.123 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.124 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.125 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.125 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.125 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.125 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.125 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.125 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.125 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T01:47:27.125 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T01:47:27.125 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.125 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.125 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.125 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.125 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.126 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.126 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.126 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.126 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.126 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.126 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.126 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T01:47:27.126 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T01:47:27.126 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.126 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.126 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.126 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.126 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.127 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.127 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.127 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.127 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.127 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.127 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.127 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T01:47:27.127 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T01:47:27.127 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.127 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.127 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.127 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.127 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.128 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.128 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.128 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.128 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.128 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.128 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.128 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T01:47:27.128 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T01:47:27.128 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.128 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.128 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.128 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.128 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.129 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.130 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.130 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.130 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.130 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.130 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.130 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T01:47:27.130 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T01:47:27.130 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.130 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.130 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.130 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T01:47:27.130 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.130 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.131 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-04-25T01:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-04-25T01:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.133 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.133 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.133 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.133 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.133 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T01:47:27.133 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T01:47:27.133 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.133 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.133 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.133 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T01:47:27.133 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.133 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.133 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.134 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.134 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.134 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.134 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.134 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T01:47:27.134 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T01:47:27.134 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.134 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.134 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.134 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.134 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.134 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.134 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.135 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.135 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.135 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.135 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.135 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T01:47:27.135 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T01:47:27.135 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.135 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.135 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.135 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.135 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.135 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.136 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.136 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.136 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.136 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.136 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.136 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T01:47:27.136 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T01:47:27.136 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.136 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.136 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.137 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.137 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.137 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.137 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.137 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.137 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.137 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.137 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.138 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T01:47:27.138 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T01:47:27.138 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.138 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.138 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.138 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.138 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.139 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.139 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.139 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.139 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.139 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.139 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.139 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T01:47:27.140 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T01:47:27.140 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.140 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.140 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.140 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.140 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.140 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.141 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.141 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.141 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.141 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.141 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.141 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T01:47:27.141 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T01:47:27.141 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.141 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.142 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.142 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.142 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.142 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.142 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.142 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.142 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.142 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.142 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.142 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T01:47:27.142 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T01:47:27.142 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.143 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.143 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.143 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.143 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.143 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.143 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.143 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.143 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.143 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.143 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.143 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T01:47:27.143 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T01:47:27.143 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.144 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.144 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.144 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.144 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.144 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.144 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.144 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.144 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.144 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.144 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.144 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T01:47:27.144 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T01:47:27.144 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.145 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.145 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.145 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.145 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.145 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.145 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.145 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.145 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.145 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.145 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.145 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T01:47:27.145 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T01:47:27.146 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.146 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.146 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.146 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.146 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.146 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.146 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.146 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.146 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.146 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.146 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.146 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T01:47:27.147 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T01:47:27.147 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.147 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.147 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.147 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.147 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.147 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.147 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.147 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.147 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.147 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.147 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.147 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T01:47:27.148 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T01:47:27.148 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.148 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.148 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.148 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.148 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.148 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.148 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.148 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.148 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.148 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.148 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.148 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T01:47:27.149 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T01:47:27.149 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.149 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.149 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.149 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.149 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.149 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.149 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.149 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.149 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.149 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.149 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.149 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T01:47:27.150 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T01:47:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T01:47:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.151 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T01:47:27.152 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T01:47:27.152 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.152 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.152 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.152 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.152 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.152 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.152 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.152 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.152 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.152 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.152 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.153 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T01:47:27.153 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T01:47:27.153 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.153 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.153 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.153 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-04-25T01:47:27.153 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.153 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.153 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.153 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.153 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.153 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.153 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.154 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T01:47:27.154 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T01:47:27.154 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.154 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.154 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.154 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-04-25T01:47:27.154 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.154 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.154 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.154 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.154 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.154 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.154 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.155 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T01:47:27.155 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T01:47:27.155 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.155 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.155 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.155 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.155 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.155 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.155 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.155 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.155 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.155 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.155 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.156 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T01:47:27.157 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T01:47:27.157 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.157 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.157 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.157 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.157 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.157 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.157 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.157 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.157 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.157 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.157 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.157 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T01:47:27.158 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T01:47:27.158 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.158 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.158 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.158 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.158 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.158 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.158 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.158 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.158 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.158 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.158 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.158 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T01:47:27.159 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T01:47:27.159 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.159 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.159 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.159 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:27.159 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.159 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.159 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.159 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.159 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.159 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.159 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.159 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T01:47:27.160 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-04-25T01:47:27.161 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.162 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.162 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.162 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.162 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.162 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.162 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.162 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.162 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.162 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.162 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.162 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-04-25T01:47:27.162 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-04-25T01:47:27.162 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.163 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.164 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.164 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.164 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.164 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.164 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.164 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-04-25T01:47:27.164 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.164 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.164 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.164 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-04-25T01:47:27.164 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-04-25T01:47:27.164 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.165 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.165 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.165 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-04-25T01:47:27.165 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.165 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.165 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.165 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.165 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.165 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.165 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.165 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-04-25T01:47:27.165 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-04-25T01:47:27.165 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.166 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.166 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.166 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.166 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.166 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.166 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.166 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.166 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.166 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.166 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.166 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.166 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.166 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.167 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.167 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.167 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.167 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.167 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.167 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.167 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.167 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.167 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.167 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.167 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.167 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.167 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.167 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.168 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.168 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.168 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.168 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.168 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.168 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.168 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.168 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.168 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.168 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.168 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.168 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.168 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.169 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.169 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.169 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.169 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.169 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.169 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.169 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.169 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.169 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.169 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.169 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.169 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.169 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.170 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.170 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.170 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.170 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.170 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.170 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.170 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.170 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.170 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.170 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.170 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.170 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.170 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.171 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.171 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.171 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.171 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.171 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.171 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.171 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.171 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.171 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.171 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.171 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-04-25T01:47:27.171 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-04-25T01:47:27.171 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.172 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.173 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.173 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.173 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.173 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.173 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.173 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.173 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.173 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.173 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.173 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T01:47:27.173 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T01:47:27.173 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.173 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.174 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.174 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-04-25T01:47:27.174 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.174 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.174 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.174 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.174 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.174 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.174 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.174 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T01:47:27.174 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T01:47:27.174 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.175 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.176 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.177 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-04-25T01:47:27.177 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.177 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.177 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.177 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.177 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.177 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.177 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.177 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-04-25T01:47:27.177 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-04-25T01:47:27.177 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.177 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.177 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.178 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-04-25T01:47:27.178 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.178 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.178 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.178 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.178 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-04-25T01:47:27.178 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.178 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.178 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.178 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.178 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.179 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-04-25T01:47:27.180 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-04-25T01:47:27.180 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.180 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.180 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.180 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.180 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.180 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.180 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.180 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.180 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.180 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.180 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.180 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.180 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.181 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-04-25T01:47:27.181 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.181 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.181 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.181 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-04-25T01:47:27.181 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-04-25T01:47:27.181 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.181 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.181 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.181 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.181 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.181 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.182 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.182 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-04-25T01:47:27.182 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.182 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.182 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.182 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.182 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.182 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.182 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.182 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.182 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.182 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.182 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.183 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.183 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.183 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.183 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.183 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.183 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.183 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.183 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.183 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.183 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.183 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.183 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.183 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.184 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.184 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.184 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.184 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.184 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.184 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.184 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.184 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.184 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.184 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.184 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.184 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.184 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.185 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.185 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.185 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.185 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.185 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.185 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.185 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.185 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.185 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.185 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.185 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.185 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.185 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.186 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.186 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.186 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.186 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.186 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.186 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.186 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.186 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.186 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.186 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.186 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.186 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.186 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.187 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.188 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.188 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T01:47:27.188 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.188 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.188 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.188 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-04-25T01:47:27.188 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-04-25T01:47:27.188 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.188 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.188 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.188 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-04-25T01:47:27.188 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.188 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.189 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.189 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T01:47:27.189 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.189 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.189 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.189 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-04-25T01:47:27.189 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-04-25T01:47:27.189 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.189 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.189 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.189 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-04-25T01:47:27.189 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.189 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.190 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.190 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-04-25T01:47:27.190 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.190 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.190 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.190 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-04-25T01:47:27.190 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-04-25T01:47:27.190 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.190 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.190 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.190 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-04-25T01:47:27.190 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.190 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.191 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.191 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T01:47:27.191 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.191 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.191 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.191 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-04-25T01:47:27.191 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-04-25T01:47:27.191 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.191 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.191 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.191 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.191 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.191 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.192 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.192 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T01:47:27.192 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.192 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.192 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.192 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:27.192 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:27.192 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.192 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.192 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.192 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:27.192 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.192 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.193 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.193 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T01:47:27.193 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.193 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.193 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.193 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-04-25T01:47:27.193 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-04-25T01:47:27.193 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.193 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.193 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.193 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-04-25T01:47:27.193 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.193 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.194 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.194 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T01:47:27.194 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.194 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.194 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.194 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.194 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.194 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-04-25T01:47:27.194 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.194 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.194 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.195 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.196 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.196 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.196 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.196 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.196 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.196 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.196 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.196 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.196 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.196 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.196 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.196 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.196 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.197 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.198 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.198 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.198 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.198 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.198 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.198 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.198 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.198 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.198 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.198 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.198 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.198 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-04-25T01:47:27.199 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-04-25T01:47:27.200 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.201 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.201 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.201 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-04-25T01:47:27.201 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.201 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.201 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.201 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.201 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.201 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.201 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.201 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:27.201 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.203 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-04-25T01:47:27.203 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-04-25T01:47:27.203 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-04-25T01:47:27.203 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.203 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-04-25T01:47:27.203 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-04-25T01:47:27.203 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.203 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.203 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.203 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-04-25T01:47:27.203 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.203 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.203 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.203 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T01:47:27.204 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.204 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.204 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.204 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-04-25T01:47:27.204 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-04-25T01:47:27.204 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.204 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.204 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.204 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-04-25T01:47:27.204 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.204 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.204 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.205 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-04-25T01:47:27.206 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.206 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.206 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.206 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:27.206 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:27.206 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.206 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.206 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.206 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-04-25T01:47:27.206 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-04-25T01:47:27.206 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.206 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.207 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T01:47:27.207 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.207 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.207 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.207 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.207 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.207 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.207 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.207 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.207 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.207 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.207 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.207 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.208 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.208 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.208 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.208 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.208 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.208 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.208 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.208 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.208 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.208 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.208 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.208 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.208 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.209 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.210 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.210 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.210 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.210 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.210 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.210 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.210 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.210 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.210 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.210 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.210 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.210 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.210 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.211 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.211 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.211 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.211 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.211 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.211 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.211 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.211 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.211 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.211 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.211 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.211 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.211 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.212 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-04-25T01:47:27.213 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.214 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.214 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.214 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-04-25T01:47:27.214 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-04-25T01:47:27.214 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.214 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.214 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.214 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-04-25T01:47:27.214 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.214 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.214 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.214 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T01:47:27.214 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.215 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.215 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.215 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-04-25T01:47:27.215 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-04-25T01:47:27.215 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.215 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.215 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.215 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T01:47:27.215 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T01:47:27.215 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-04-25T01:47:27.215 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.215 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T01:47:27.215 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.216 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.217 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.217 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-04-25T01:47:27.217 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-04-25T01:47:27.217 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.217 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.217 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.217 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-04-25T01:47:27.217 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.217 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.217 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.217 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T01:47:27.217 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.217 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.218 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.218 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.218 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.218 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.218 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-04-25T01:47:27.218 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.218 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.218 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.218 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.218 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.218 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.218 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.219 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.219 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.221 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.221 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.221 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.222 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.222 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.222 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.222 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.222 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.222 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.222 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.222 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.222 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.222 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.222 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.222 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.223 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.223 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.223 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.223 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.223 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.223 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.223 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.223 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.223 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.223 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.223 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.223 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.224 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.225 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.225 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.225 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.225 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.225 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.225 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.225 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.225 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.225 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.225 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.225 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.225 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.225 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.226 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.226 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.226 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.226 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.226 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.226 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.226 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.226 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-04-25T01:47:27.226 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.226 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.226 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.227 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.228 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.229 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.230 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.230 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.230 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.230 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.230 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.230 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.230 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.230 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.230 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.230 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.230 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.230 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.231 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.231 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.231 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.231 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.231 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.231 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.231 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.231 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.231 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.231 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.231 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.231 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.231 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.232 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-04-25T01:47:27.232 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.232 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.232 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.232 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-04-25T01:47:27.232 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-04-25T01:47:27.232 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.232 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.232 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.232 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-04-25T01:47:27.232 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.232 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.232 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.233 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T01:47:27.233 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.233 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.233 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.233 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-04-25T01:47:27.233 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-04-25T01:47:27.233 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.233 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.233 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.233 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.233 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-04-25T01:47:27.233 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.233 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.234 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T01:47:27.234 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.234 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.234 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.234 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.234 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.234 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.234 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.234 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.234 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.234 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.234 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.235 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.236 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.237 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.237 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.237 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.237 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.237 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.237 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.237 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.237 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.237 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.237 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.237 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.237 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.237 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.237 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.238 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.238 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.238 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.238 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.238 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.238 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.238 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.238 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.238 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.238 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.238 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.238 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.239 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.239 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.239 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.239 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.239 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.239 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.239 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.239 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-04-25T01:47:27.239 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.239 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.239 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-04-25T01:47:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-04-25T01:47:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-04-25T01:47:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-04-25T01:47:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T01:47:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.241 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.241 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.241 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.241 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.241 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.241 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.241 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.241 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.241 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.241 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.241 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.241 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.241 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.241 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.242 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.242 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.242 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.242 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.242 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.242 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.242 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.242 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.242 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.242 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.242 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.242 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.243 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.243 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.243 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.243 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.243 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.243 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.243 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.243 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.243 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.243 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.243 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.243 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.243 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.244 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.245 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.245 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.245 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.245 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.245 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.245 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.245 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.245 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.245 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.245 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.245 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.245 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.245 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.246 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-04-25T01:47:27.246 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-04-25T01:47:27.246 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.246 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.246 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.246 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.246 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.246 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.246 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.246 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-04-25T01:47:27.246 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T01:47:27.246 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.246 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.247 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-04-25T01:47:27.247 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-04-25T01:47:27.247 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.247 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.247 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.247 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-04-25T01:47:27.247 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.247 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.247 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.247 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-04-25T01:47:27.247 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.247 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.247 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.248 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-04-25T01:47:27.248 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-04-25T01:47:27.248 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.248 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.248 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.248 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-04-25T01:47:27.248 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.248 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.248 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.248 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T01:47:27.248 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.249 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.249 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.249 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-04-25T01:47:27.249 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-04-25T01:47:27.249 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.249 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.249 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.249 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.249 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.249 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.249 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.249 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-04-25T01:47:27.250 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.250 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.250 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.250 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-04-25T01:47:27.250 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-04-25T01:47:27.250 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.250 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.250 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.250 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-04-25T01:47:27.250 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.250 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.250 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.251 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T01:47:27.251 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T01:47:27.251 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.251 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.251 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.251 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.251 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-04-25T01:47:27.251 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.251 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.251 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.251 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.251 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.252 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.253 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.253 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.253 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.253 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.253 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.253 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.253 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.253 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.253 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.253 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.253 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.253 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.253 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.254 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.254 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.254 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.254 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.254 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.254 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.254 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.254 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.254 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.254 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.254 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.254 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.254 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.256 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.256 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.256 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.256 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.256 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.256 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.256 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.256 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.256 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.256 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.256 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.256 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.257 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.258 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.258 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.258 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.258 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.258 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.258 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.258 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.258 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.258 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.258 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.258 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.258 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.259 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.259 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.259 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.259 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.259 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.259 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.259 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.259 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.259 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.259 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.259 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.259 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.259 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.260 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.260 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.260 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.260 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.260 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.260 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.260 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.260 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.260 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.260 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.260 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.260 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.260 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.260 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.261 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.262 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.262 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.262 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.262 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-04-25T01:47:27.262 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.262 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.262 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.262 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.262 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.262 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.262 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.263 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.263 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.263 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.263 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.263 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.263 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.263 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.263 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.263 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.263 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.263 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.263 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.263 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.264 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.265 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.266 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.266 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.266 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.266 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.266 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.266 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.266 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.266 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.266 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.266 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.266 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.266 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.267 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.267 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.267 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.267 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.267 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.267 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.267 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.267 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.267 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.267 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.267 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.267 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.268 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-04-25T01:47:27.268 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.268 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.268 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.268 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-04-25T01:47:27.268 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-04-25T01:47:27.268 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.268 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.268 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.268 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.268 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.268 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.268 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.268 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T01:47:27.269 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.269 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.269 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.269 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.269 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.269 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.269 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.269 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.269 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.269 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.269 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.269 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.269 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.270 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.270 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.270 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.270 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.270 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.270 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.270 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.270 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.270 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.270 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.270 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.270 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.270 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.271 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.271 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.271 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.271 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.271 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.271 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.271 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.271 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.271 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.271 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.271 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.271 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.271 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.272 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.273 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.274 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.274 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.274 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.274 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-04-25T01:47:27.274 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-04-25T01:47:27.274 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.274 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.274 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.274 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.274 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.274 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.274 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.274 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.275 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.276 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.276 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.276 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.276 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.276 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.276 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.276 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.276 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.276 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.276 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.276 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.276 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.276 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.277 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.277 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.277 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.277 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.277 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.277 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.277 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.277 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.277 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.277 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.277 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.277 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.277 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.278 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.278 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.278 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.278 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.278 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.278 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.278 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.278 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.278 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.278 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.278 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.278 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.278 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.279 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.279 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.279 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.279 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.279 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.279 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.279 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.279 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.279 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.279 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.279 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.279 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.279 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.280 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.280 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.280 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.280 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.280 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.280 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.280 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.280 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.280 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.280 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.280 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.280 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.280 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.281 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.281 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.281 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.281 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-04-25T01:47:27.281 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.281 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.281 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.281 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.281 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.281 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.281 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.282 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.282 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.282 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.282 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.282 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.282 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.282 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.282 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.282 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.282 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.282 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.282 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.282 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.282 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.283 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.283 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.283 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.283 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.283 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.283 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.283 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.283 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.283 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.283 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.283 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.283 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.283 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.284 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.284 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.284 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.284 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.284 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.284 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.284 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.284 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.284 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.284 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.284 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.284 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.284 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.285 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.286 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.286 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.286 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.286 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.286 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.286 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.286 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.286 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.286 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.286 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.286 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.287 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.288 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.289 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.289 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.289 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.289 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.289 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.289 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.289 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.289 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.289 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.289 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.289 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.289 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.289 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.290 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.290 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.290 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.290 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.290 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.290 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.290 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.290 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.290 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.290 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.290 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.290 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.290 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.291 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.292 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.293 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-04-25T01:47:27.293 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-04-25T01:47:27.293 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:27.293 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.293 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.293 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-04-25T01:47:27.293 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-04-25T01:47:27.293 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.293 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.293 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-04-25T01:47:27.293 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T01:47:27.293 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.293 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.294 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.294 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.294 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-04-25T01:47:27.294 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.294 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.294 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.294 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-04-25T01:47:27.294 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-04-25T01:47:27.294 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.294 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.294 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.295 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.295 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.295 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.295 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.295 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T01:47:27.295 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.295 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.295 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.295 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-04-25T01:47:27.295 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-04-25T01:47:27.295 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.295 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.295 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.296 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.297 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.297 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.297 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.297 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.297 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.297 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.297 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.297 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.297 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.297 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.297 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.297 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.297 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.297 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.298 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.298 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.298 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.298 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.298 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.298 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.298 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.298 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.298 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.298 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.298 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.298 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.298 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.299 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.299 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.299 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.299 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.299 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.299 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.299 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.299 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.299 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.299 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.299 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.299 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.299 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.300 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.301 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.302 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.302 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-04-25T01:47:27.302 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.302 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.302 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.302 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:27.302 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:27.302 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.302 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.302 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.302 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T01:47:27.302 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.303 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-04-25T01:47:27.304 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-04-25T01:47:27.304 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.304 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.304 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.304 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.304 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.304 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.304 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.304 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.304 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.304 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.304 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.305 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-04-25T01:47:27.305 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-04-25T01:47:27.305 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.305 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.305 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.305 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.305 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.305 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.305 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.305 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T01:47:27.305 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T01:47:27.305 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.305 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.306 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.306 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.306 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.306 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.306 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.306 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.306 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.306 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.306 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.306 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.306 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.306 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.306 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.307 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.307 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.307 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.307 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.307 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.307 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.307 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.307 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.307 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.307 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.307 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.307 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.307 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.308 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.308 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.308 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.308 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.308 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.308 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.308 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.308 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.308 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.308 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.308 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.308 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.308 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.309 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.309 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.309 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.309 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.309 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.309 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.309 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.309 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.309 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.309 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.309 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.309 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.309 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.309 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.310 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.310 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.310 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.310 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.310 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.310 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.310 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.310 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.310 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.310 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.310 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.310 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.310 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.311 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-04-25T01:47:27.311 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-04-25T01:47:27.311 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.311 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.311 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.311 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.311 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.311 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.311 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.311 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.311 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.311 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.311 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.312 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T01:47:27.312 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T01:47:27.312 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.312 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.312 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.312 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-04-25T01:47:27.312 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.312 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.312 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.312 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.312 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.312 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.312 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.313 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-04-25T01:47:27.313 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-04-25T01:47:27.313 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:27.313 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.313 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.313 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-04-25T01:47:27.313 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.313 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.313 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.313 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.313 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.313 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.313 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T01:47:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T01:47:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-04-25T01:47:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T01:47:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.314 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.315 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T01:47:27.315 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T01:47:27.315 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.315 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.315 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.315 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-04-25T01:47:27.315 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.315 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.315 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.315 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T01:47:27.315 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.315 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.315 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.316 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-04-25T01:47:27.316 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-04-25T01:47:27.316 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.316 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.316 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.316 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-04-25T01:47:27.316 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.316 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.316 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.316 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.316 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.316 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.316 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.317 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-04-25T01:47:27.317 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-04-25T01:47:27.317 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.317 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.317 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.317 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-04-25T01:47:27.317 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.317 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.317 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.317 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-04-25T01:47:27.317 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-04-25T01:47:27.317 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.318 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.318 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.318 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.318 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.318 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-04-25T01:47:27.318 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-04-25T01:47:27.318 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.318 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.318 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.318 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-04-25T01:47:27.318 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-04-25T01:47:27.318 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-04-25T01:47:27.318 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.319 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.319 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.319 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.319 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.319 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.319 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.319 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-04-25T01:47:27.319 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.319 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.319 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.319 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.319 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.320 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.321 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.321 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.321 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.321 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.321 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.321 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.321 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.321 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.321 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.321 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.321 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.321 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.321 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.322 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.322 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.322 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.322 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.322 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.322 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.322 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.322 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.322 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.322 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.322 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.322 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.322 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.323 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.324 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.324 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.324 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.324 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.324 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.324 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.324 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.324 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.324 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.324 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-04-25T01:47:27.324 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-04-25T01:47:27.324 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.324 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.325 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.326 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.326 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.326 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.326 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.326 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.326 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.326 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.326 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.326 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-04-25T01:47:27.326 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-04-25T01:47:27.326 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.326 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.326 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.327 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.327 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.327 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.327 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.327 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.327 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.327 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.327 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.327 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.327 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.327 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-04-25T01:47:27.327 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.328 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.329 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-04-25T01:47:27.329 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-04-25T01:47:27.329 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.329 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.329 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.329 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.329 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.329 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.329 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.329 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.329 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.329 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.329 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.330 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.330 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.330 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.330 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.330 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.330 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.330 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.330 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.330 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.330 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.330 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.330 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.330 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.331 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.331 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.331 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.331 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.331 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.331 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.331 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.331 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.331 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.331 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.331 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.331 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.331 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.332 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.332 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.332 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.332 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.332 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.332 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.332 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.332 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.332 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.332 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.332 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.332 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.333 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.333 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.333 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.333 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.333 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.333 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.333 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.333 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.333 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.333 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.333 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.333 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.333 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.333 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.334 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.334 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.334 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.334 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.334 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.334 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.334 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.334 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.334 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.334 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.334 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.334 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.334 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.335 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T01:47:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T01:47:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.336 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.337 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.337 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.337 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.337 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-04-25T01:47:27.337 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.337 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.337 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.337 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.337 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.337 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.337 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.337 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.337 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.338 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.338 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.338 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.338 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.338 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.338 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.338 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.338 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.338 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.338 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.338 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.338 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.338 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.338 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.340 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.341 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.341 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.341 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.341 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.341 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.341 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.341 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.341 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.341 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.341 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.341 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.341 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.341 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.342 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.342 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.342 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.342 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.342 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.342 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.342 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.342 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-04-25T01:47:27.342 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.342 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.342 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.342 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.343 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.344 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.344 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.344 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.344 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.344 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.344 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.344 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.344 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.344 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.344 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.344 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.344 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.344 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.346 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.346 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.346 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.346 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.346 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.346 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.346 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.346 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.346 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.346 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.346 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.346 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.347 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.347 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.347 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.347 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.347 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.347 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.347 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.347 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.347 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.347 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.347 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.347 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.347 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-04-25T01:47:27.348 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-04-25T01:47:27.348 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.348 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.348 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.348 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-04-25T01:47:27.348 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.348 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.348 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.348 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T01:47:27.348 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.348 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.348 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.348 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.348 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.349 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-04-25T01:47:27.349 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.349 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.349 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.349 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.349 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.349 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.349 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.349 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.349 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.349 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.349 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.350 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.350 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.350 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.350 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.350 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.350 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.350 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.350 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.350 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.350 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.350 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.350 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.350 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.351 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.351 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.351 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.351 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.351 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.351 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.351 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.351 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.351 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.351 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.351 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.351 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.351 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.352 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.353 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.353 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.353 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.353 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.353 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.353 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.353 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.353 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.353 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.353 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.353 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.353 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.353 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.354 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.354 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.354 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.354 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.354 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-04-25T01:47:27.354 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-04-25T01:47:27.354 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.354 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.354 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.354 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.354 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.354 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.354 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.355 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.355 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.355 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.355 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.355 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-04-25T01:47:27.355 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-04-25T01:47:27.355 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.355 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.355 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.355 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-04-25T01:47:27.355 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.355 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.355 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.356 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.356 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.356 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.356 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.356 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-04-25T01:47:27.356 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-04-25T01:47:27.356 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.356 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.356 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.356 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.356 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.356 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.356 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.357 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.357 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.357 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.357 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.357 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-04-25T01:47:27.357 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-04-25T01:47:27.357 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.357 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.357 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.357 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.357 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.357 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.357 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.358 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.358 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.358 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.358 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.358 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-04-25T01:47:27.358 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-04-25T01:47:27.358 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:27.358 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.358 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.358 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.358 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.358 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.359 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.360 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.360 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.360 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.360 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.360 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-04-25T01:47:27.360 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-04-25T01:47:27.360 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.360 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.360 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.360 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.360 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.360 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.360 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.361 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.361 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.361 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.361 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.361 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-04-25T01:47:27.361 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-04-25T01:47:27.361 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.361 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.361 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.361 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.361 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.362 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.362 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.362 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.362 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.362 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.362 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.362 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-04-25T01:47:27.362 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-04-25T01:47:27.362 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.362 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.362 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.362 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.362 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T01:47:27.363 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-04-25T01:47:27.363 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.363 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.363 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.363 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.363 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.363 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-04-25T01:47:27.363 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-04-25T01:47:27.363 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.363 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.363 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.363 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.364 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.364 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.364 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.364 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.364 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.364 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.364 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.364 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-04-25T01:47:27.364 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-04-25T01:47:27.364 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.364 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.365 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.365 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.365 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.365 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.365 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.365 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.365 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.365 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.365 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.365 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-04-25T01:47:27.365 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-04-25T01:47:27.365 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.365 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.366 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-04-25T01:47:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-04-25T01:47:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-04-25T01:47:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T01:47:27.367 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.368 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.368 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.368 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-04-25T01:47:27.368 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-04-25T01:47:27.368 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.368 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.368 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.368 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.368 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.368 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.368 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.368 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-04-25T01:47:27.368 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.369 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.369 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.369 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.369 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.369 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.369 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.369 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.369 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.369 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.369 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.369 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.369 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.369 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.370 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.370 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.370 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.370 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.370 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.370 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.370 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.370 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.370 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.370 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.370 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.370 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.371 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.372 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.372 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.372 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.372 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.372 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.372 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.372 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.372 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.372 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.372 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.372 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.372 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.372 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.373 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.373 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.373 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.373 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.373 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.373 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.373 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.373 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.373 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.373 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.373 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.373 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-04-25T01:47:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.375 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.375 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.375 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.375 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.375 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.375 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.375 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.375 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.375 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.375 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.375 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.375 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.375 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.375 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.377 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.377 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.377 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.377 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.377 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.377 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.377 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.377 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.377 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.377 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.377 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.377 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.377 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.378 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.378 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.378 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.378 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.378 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.378 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.378 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.378 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.378 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.378 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.378 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.378 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.378 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.379 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.379 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.379 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.379 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.379 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.379 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.379 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.379 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.379 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.379 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.379 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.379 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.379 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.380 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.380 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-04-25T01:47:27.380 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.380 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.380 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.380 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.380 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.380 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.380 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.380 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.380 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.380 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.380 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.380 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.381 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.382 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.382 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.382 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.382 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.382 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.382 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.382 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.382 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.382 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.382 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.382 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.382 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.383 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.384 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.384 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.384 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.384 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.384 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.384 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.384 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.384 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.384 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.384 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.384 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.384 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.384 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.385 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.386 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.386 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T01:47:27.386 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.386 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.386 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.386 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.386 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.386 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.386 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.386 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:27.386 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:27.386 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.387 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.387 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.387 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-04-25T01:47:27.387 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.387 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.387 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.387 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.387 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.387 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.387 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.387 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.388 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.388 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.388 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.388 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.388 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.388 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.388 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.388 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.388 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.388 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.388 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.388 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.388 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.389 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.389 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.389 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.389 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.389 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.389 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.389 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.389 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.389 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.389 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.389 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.389 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.389 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.390 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.390 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.390 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.390 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.390 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.390 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.390 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.390 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.390 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.390 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.390 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.390 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.391 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.391 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.391 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.391 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.391 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.391 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.391 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.391 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.391 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.391 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.391 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.391 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.391 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.392 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.392 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.392 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.392 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.392 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.392 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.392 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.392 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.392 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.392 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.392 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.392 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.392 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.393 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.393 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.393 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.393 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.393 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-04-25T01:47:27.393 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.393 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.393 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.393 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-04-25T01:47:27.393 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-04-25T01:47:27.393 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.393 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.393 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.394 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.394 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.394 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.394 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.394 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T01:47:27.394 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.394 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.394 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.394 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-04-25T01:47:27.394 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-04-25T01:47:27.394 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.395 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.395 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.395 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.395 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.395 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.395 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.395 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T01:47:27.395 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.395 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.395 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.395 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-04-25T01:47:27.395 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-04-25T01:47:27.395 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.396 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.397 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.397 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.397 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.397 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.397 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.397 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T01:47:27.397 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.397 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.397 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.397 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-04-25T01:47:27.397 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-04-25T01:47:27.397 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.397 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.398 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.399 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.400 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.400 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.400 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.400 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.400 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.400 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.400 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.400 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-04-25T01:47:27.400 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-04-25T01:47:27.400 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.400 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.400 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.401 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.402 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.402 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.402 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.402 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.402 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.402 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.402 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.402 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:27.402 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:27.402 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.402 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.402 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.403 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-04-25T01:47:27.403 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-04-25T01:47:27.403 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.403 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.403 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.403 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.403 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.403 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.403 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-04-25T01:47:27.403 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-04-25T01:47:27.403 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.403 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.404 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.405 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.405 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.405 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.405 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.405 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.405 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.405 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.405 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.405 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-04-25T01:47:27.405 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-04-25T01:47:27.405 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.405 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.405 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.406 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.406 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.406 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.406 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.406 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.406 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.406 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.406 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.406 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.406 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.406 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.406 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.406 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.406 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.407 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.407 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.407 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.407 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.407 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.407 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.407 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.407 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.407 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.407 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.407 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.407 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.407 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.407 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.408 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.409 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.409 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.409 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.409 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.409 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.409 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.409 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.409 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.409 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.409 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.409 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.409 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.409 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.410 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.413 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.413 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.413 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.413 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.413 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.413 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-04-25T01:47:27.413 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-04-25T01:47:27.413 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.413 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.413 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.413 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T01:47:27.413 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.413 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.414 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.414 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.414 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.414 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.414 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.414 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.414 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.414 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.414 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.414 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-04-25T01:47:27.414 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.414 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.414 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.415 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.415 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.415 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.415 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.415 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.415 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.415 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.415 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.415 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.415 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.415 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.415 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.415 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.415 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.416 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.416 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.416 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.416 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.416 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.416 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.416 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.416 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.416 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.416 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.416 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.416 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.416 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.417 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.418 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.418 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.418 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.418 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.418 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.418 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.418 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.418 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.418 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.418 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.418 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.418 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.418 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.419 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.419 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.419 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.419 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.419 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.419 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.419 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.419 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.419 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.419 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.419 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.419 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.419 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.419 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.420 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.420 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.420 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-04-25T01:47:27.420 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.420 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.420 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.420 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.420 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.420 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.420 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.420 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.420 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.420 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.421 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.422 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.422 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.422 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.422 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.422 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.422 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.422 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.422 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.422 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.422 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.425 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.425 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.426 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.426 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.426 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.426 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.426 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.426 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.426 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.426 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.426 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.426 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.426 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.426 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.426 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.427 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.427 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.427 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.427 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.427 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.427 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.427 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.427 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.427 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.427 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.427 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.427 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.427 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-04-25T01:47:27.428 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.429 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.429 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.429 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T01:47:27.429 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.429 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.429 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.429 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-04-25T01:47:27.429 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-04-25T01:47:27.429 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.429 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.429 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.429 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.429 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.430 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.430 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.430 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-04-25T01:47:27.430 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.430 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.430 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.430 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-04-25T01:47:27.430 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-04-25T01:47:27.430 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.430 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.430 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.430 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:27.430 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.431 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.431 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.431 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T01:47:27.431 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.431 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.431 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.431 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-04-25T01:47:27.431 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T01:47:27.431 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.431 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.431 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.431 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T01:47:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-04-25T01:47:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.432 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.433 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.433 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-04-25T01:47:27.433 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-04-25T01:47:27.433 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:27.433 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.433 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.433 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-04-25T01:47:27.433 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.433 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.433 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.433 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.433 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.433 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.433 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.434 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-04-25T01:47:27.434 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-04-25T01:47:27.434 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.434 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.434 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.434 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.434 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.434 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.434 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.434 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.434 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.434 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.434 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.435 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:27.435 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:27.435 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.435 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.435 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.435 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:27.435 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.435 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.435 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.435 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.435 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.435 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.435 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.436 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.436 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.436 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.436 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.436 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.436 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.436 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.436 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.436 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.436 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.436 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.436 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.436 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.437 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.438 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.438 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.438 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.438 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.438 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.438 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.438 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.438 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.438 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.438 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.438 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.438 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.438 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.439 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.439 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.439 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.439 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.439 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.439 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.439 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.439 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.439 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.439 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.439 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.439 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.439 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.440 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.440 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.440 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.440 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.440 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.440 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.440 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.440 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.440 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.440 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.440 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.440 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.440 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.441 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-04-25T01:47:27.441 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-04-25T01:47:27.441 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.441 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.441 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.441 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.441 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.441 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.441 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.441 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.441 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.441 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.441 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.442 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-04-25T01:47:27.442 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-04-25T01:47:27.442 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.442 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.442 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.442 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-04-25T01:47:27.442 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.442 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.442 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.442 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.442 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.442 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.442 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.443 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-04-25T01:47:27.443 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-04-25T01:47:27.443 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.443 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.443 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.443 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T01:47:27.443 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.443 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.443 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.443 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.443 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.443 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.443 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.444 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.444 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.444 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.444 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T01:47:27.444 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.444 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.444 INFO:teuthology.orchestra.run.smithi078.stdout: "modules": [ 2024-04-25T01:47:27.444 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-04-25T01:47:27.444 INFO:teuthology.orchestra.run.smithi078.stdout: "dashboard", 2024-04-25T01:47:27.444 INFO:teuthology.orchestra.run.smithi078.stdout: "iostat", 2024-04-25T01:47:27.444 INFO:teuthology.orchestra.run.smithi078.stdout: "nfs", 2024-04-25T01:47:27.444 INFO:teuthology.orchestra.run.smithi078.stdout: "restful" 2024-04-25T01:47:27.444 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.445 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-04-25T01:47:27.445 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.445 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-04-25T01:47:27.445 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.445 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.445 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.445 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:27.445 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:27.445 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.445 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.445 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.445 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:27.445 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.446 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.447 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.447 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.447 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.447 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.447 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.447 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.447 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.447 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.447 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.447 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.447 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.447 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.448 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.448 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.448 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.448 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.448 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.448 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.448 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.448 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.448 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.448 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.448 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.448 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.448 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.449 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.449 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.449 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.449 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.449 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.449 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.449 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.449 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.449 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.449 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.449 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.449 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.449 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.449 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.450 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.450 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.450 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.450 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.450 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.450 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.450 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.450 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.450 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.450 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.450 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.450 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-04-25T01:47:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-04-25T01:47:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.451 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.452 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.452 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.452 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.452 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-04-25T01:47:27.452 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.452 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.452 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.452 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-04-25T01:47:27.452 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-04-25T01:47:27.452 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.452 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.452 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.452 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-04-25T01:47:27.453 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.453 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.453 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.453 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-04-25T01:47:27.453 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.453 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.453 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.453 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-04-25T01:47:27.453 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-04-25T01:47:27.453 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.453 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.453 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.453 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.454 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.454 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.454 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.454 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-04-25T01:47:27.454 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.454 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.454 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.454 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-04-25T01:47:27.454 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-04-25T01:47:27.454 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.454 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.454 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.454 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-04-25T01:47:27.455 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.456 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.456 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.456 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-04-25T01:47:27.456 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.456 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.456 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.456 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-04-25T01:47:27.456 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-04-25T01:47:27.456 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.456 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.456 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.456 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.456 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.458 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.458 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T01:47:27.458 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.458 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.458 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.458 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-04-25T01:47:27.458 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-04-25T01:47:27.458 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.458 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.458 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.458 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.458 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.458 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.459 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.459 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-04-25T01:47:27.459 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.459 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.459 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.459 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.459 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.459 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-04-25T01:47:27.459 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.459 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.459 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.459 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-04-25T01:47:27.460 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-04-25T01:47:27.461 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.461 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.461 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.461 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-04-25T01:47:27.461 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.461 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.461 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.461 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T01:47:27.461 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T01:47:27.461 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.461 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.461 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-04-25T01:47:27.461 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-04-25T01:47:27.462 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:27.462 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.462 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.462 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:27.462 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-04-25T01:47:27.462 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-04-25T01:47:27.462 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.462 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T01:47:27.462 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T01:47:27.462 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.462 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.462 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-04-25T01:47:27.463 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-04-25T01:47:27.463 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:27.463 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.463 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.463 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-04-25T01:47:27.463 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T01:47:27.463 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-04-25T01:47:27.463 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.463 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T01:47:27.463 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.463 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.463 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.463 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-04-25T01:47:27.464 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-04-25T01:47:27.464 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.464 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.464 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.464 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T01:47:27.464 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.464 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.464 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.464 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T01:47:27.464 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T01:47:27.464 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.464 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.464 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-04-25T01:47:27.465 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-04-25T01:47:27.465 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:27.465 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.465 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.465 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-04-25T01:47:27.465 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-04-25T01:47:27.465 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-04-25T01:47:27.465 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.465 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-04-25T01:47:27.465 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T01:47:27.465 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.465 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.465 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-04-25T01:47:27.466 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-04-25T01:47:27.466 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.466 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.466 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.466 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-04-25T01:47:27.466 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.466 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.466 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.466 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T01:47:27.466 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T01:47:27.466 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.466 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.466 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-04-25T01:47:27.467 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-04-25T01:47:27.467 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:27.467 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.467 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.467 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:27.467 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-04-25T01:47:27.467 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-04-25T01:47:27.467 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.467 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T01:47:27.467 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T01:47:27.467 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.467 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.467 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.468 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.469 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.469 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.469 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.469 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.469 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.469 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.469 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.469 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.469 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.469 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.469 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.469 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.469 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.470 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.470 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.470 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.470 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.470 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.470 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.470 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.470 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.470 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.470 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.470 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.470 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.471 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.471 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.471 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.471 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.471 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.471 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.471 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.471 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.471 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.471 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.471 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.471 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.471 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.472 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.472 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.472 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.472 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.472 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.472 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.472 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.472 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.472 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.472 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.472 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.472 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.472 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-04-25T01:47:27.473 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-04-25T01:47:27.473 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:27.473 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.473 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.473 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:27.473 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.473 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.473 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.473 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T01:47:27.473 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.473 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.473 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.473 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-04-25T01:47:27.474 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-04-25T01:47:27.474 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.474 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.474 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.474 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-04-25T01:47:27.474 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.474 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.474 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.474 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-04-25T01:47:27.474 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-04-25T01:47:27.474 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-04-25T01:47:27.474 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-04-25T01:47:27.474 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-04-25T01:47:27.474 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.475 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-04-25T01:47:27.475 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.475 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.475 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.475 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-04-25T01:47:27.475 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-04-25T01:47:27.475 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.475 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.475 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.475 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.475 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.475 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T01:47:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.476 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.477 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.477 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T01:47:27.477 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.477 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.477 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.477 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-04-25T01:47:27.477 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-04-25T01:47:27.477 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.477 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.477 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.477 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T01:47:27.477 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T01:47:27.478 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.478 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.478 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T01:47:27.478 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T01:47:27.478 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.478 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.478 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-04-25T01:47:27.478 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-04-25T01:47:27.478 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:27.478 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.478 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.478 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.479 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.479 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.479 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.479 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T01:47:27.479 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.479 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.479 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.479 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.479 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.479 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-04-25T01:47:27.479 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.479 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.480 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.480 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-04-25T01:47:27.480 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-04-25T01:47:27.480 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:27.480 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.480 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.480 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-04-25T01:47:27.480 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.480 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.480 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.480 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T01:47:27.480 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.480 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.481 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.481 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-04-25T01:47:27.481 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-04-25T01:47:27.481 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.481 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.481 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.481 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-04-25T01:47:27.481 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.481 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.481 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.481 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T01:47:27.481 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.481 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.482 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.482 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-04-25T01:47:27.482 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-04-25T01:47:27.482 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.482 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.482 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.482 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-04-25T01:47:27.482 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.482 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.482 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.482 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T01:47:27.482 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.482 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.483 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.483 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-04-25T01:47:27.483 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-04-25T01:47:27.483 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.483 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.483 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.483 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.483 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.483 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.483 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.483 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T01:47:27.483 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T01:47:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-04-25T01:47:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-04-25T01:47:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T01:47:27.484 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-04-25T01:47:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T01:47:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-04-25T01:47:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T01:47:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-04-25T01:47:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-04-25T01:47:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-04-25T01:47:27.486 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.487 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.488 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.488 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T01:47:27.488 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.488 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.488 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.488 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-04-25T01:47:27.488 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-04-25T01:47:27.488 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.488 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.488 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.488 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.488 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.488 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.489 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.489 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T01:47:27.489 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.489 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.489 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.489 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-04-25T01:47:27.489 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-04-25T01:47:27.489 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.489 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.489 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.489 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.489 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.489 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.490 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.490 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T01:47:27.490 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.490 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.490 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.490 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-04-25T01:47:27.490 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-04-25T01:47:27.490 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.490 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.490 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.490 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T01:47:27.490 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.490 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.491 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.492 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-04-25T01:47:27.492 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.492 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.492 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.492 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-04-25T01:47:27.492 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-04-25T01:47:27.492 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.492 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.492 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.492 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T01:47:27.492 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.492 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.493 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.493 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-04-25T01:47:27.493 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.493 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.493 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.493 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-04-25T01:47:27.493 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-04-25T01:47:27.493 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.493 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.493 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.493 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T01:47:27.493 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.493 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.494 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.494 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T01:47:27.494 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.494 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.494 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.494 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-04-25T01:47:27.494 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-04-25T01:47:27.494 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.494 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.494 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.494 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T01:47:27.494 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.494 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.495 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.496 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-04-25T01:47:27.496 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.496 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.496 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.496 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-04-25T01:47:27.496 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-04-25T01:47:27.496 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.496 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.496 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.496 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T01:47:27.496 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.496 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.496 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.497 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-04-25T01:47:27.498 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.498 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.498 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.498 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-04-25T01:47:27.498 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-04-25T01:47:27.498 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.498 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.498 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.498 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T01:47:27.498 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.498 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.498 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.499 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-04-25T01:47:27.500 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.500 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.500 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.500 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-04-25T01:47:27.500 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-04-25T01:47:27.500 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.500 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.500 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.500 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T01:47:27.500 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.500 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.500 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.500 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T01:47:27.501 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.501 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.501 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.501 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-04-25T01:47:27.501 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-04-25T01:47:27.501 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.501 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.501 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.501 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T01:47:27.501 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.501 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.501 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.501 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-04-25T01:47:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-04-25T01:47:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-04-25T01:47:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T01:47:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-04-25T01:47:27.503 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.503 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.503 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.503 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-04-25T01:47:27.503 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-04-25T01:47:27.503 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.503 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.503 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.503 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T01:47:27.503 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.503 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.503 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.503 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-04-25T01:47:27.504 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.504 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.504 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.504 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-04-25T01:47:27.504 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-04-25T01:47:27.504 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.504 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.504 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.504 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T01:47:27.504 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.504 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.504 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.504 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-04-25T01:47:27.505 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.505 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.505 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.505 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-04-25T01:47:27.505 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-04-25T01:47:27.505 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.505 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.505 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.505 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T01:47:27.505 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.505 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.505 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.506 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-04-25T01:47:27.506 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.506 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.506 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.506 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-04-25T01:47:27.506 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-04-25T01:47:27.506 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.506 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.506 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.506 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.506 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.506 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.507 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.507 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T01:47:27.507 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.507 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.507 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.507 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-04-25T01:47:27.507 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-04-25T01:47:27.507 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.507 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.507 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.507 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:27.507 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.508 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.508 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.508 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T01:47:27.508 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.508 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.508 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.508 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-04-25T01:47:27.508 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T01:47:27.508 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.508 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.508 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.508 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-04-25T01:47:27.508 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.509 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.509 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.509 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T01:47:27.509 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.509 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.509 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.509 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-04-25T01:47:27.509 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-04-25T01:47:27.509 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.509 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.509 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.509 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-04-25T01:47:27.509 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.510 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.510 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.510 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T01:47:27.510 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.510 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.510 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.510 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-04-25T01:47:27.510 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-04-25T01:47:27.510 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.510 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.510 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.510 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-04-25T01:47:27.511 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.511 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.511 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.511 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-04-25T01:47:27.511 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.511 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.511 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.511 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-04-25T01:47:27.511 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-04-25T01:47:27.511 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.511 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.511 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.511 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.512 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.512 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.512 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.512 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T01:47:27.512 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.512 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.512 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.512 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-04-25T01:47:27.512 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-04-25T01:47:27.512 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.512 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.512 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.512 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:27.513 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.513 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.513 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.513 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-04-25T01:47:27.513 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.513 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.513 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.513 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-04-25T01:47:27.513 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-04-25T01:47:27.513 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.513 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.513 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.513 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T01:47:27.514 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.514 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.514 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.514 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T01:47:27.514 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.514 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.514 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.514 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-04-25T01:47:27.514 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-04-25T01:47:27.514 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.514 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.514 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.514 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.515 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.516 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.516 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.516 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T01:47:27.516 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.516 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.516 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.516 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-04-25T01:47:27.516 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-04-25T01:47:27.516 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.516 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.516 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.516 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.516 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T01:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.518 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-04-25T01:47:27.519 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-04-25T01:47:27.519 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.519 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.519 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.519 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.519 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.519 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.519 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.519 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T01:47:27.519 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.519 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.519 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.519 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.520 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.521 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.521 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.521 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.521 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.521 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.521 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.521 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.521 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.521 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.521 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.521 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.521 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.521 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.522 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.522 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.522 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.522 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.522 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.522 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.522 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.522 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.522 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.522 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.522 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.522 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.522 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.523 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.524 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T01:47:27.524 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.524 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.524 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.524 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T01:47:27.524 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T01:47:27.524 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.524 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.524 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.524 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-04-25T01:47:27.524 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.524 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.525 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.525 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T01:47:27.525 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.525 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.525 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.525 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-04-25T01:47:27.525 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-04-25T01:47:27.525 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.525 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.525 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.525 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.525 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.526 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.526 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.526 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-04-25T01:47:27.526 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.526 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.526 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.526 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-04-25T01:47:27.526 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-04-25T01:47:27.526 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.526 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.526 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.526 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.527 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.527 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.527 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.527 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T01:47:27.527 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.527 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.527 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.527 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-04-25T01:47:27.527 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-04-25T01:47:27.527 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.527 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.527 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.527 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-04-25T01:47:27.528 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.529 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.530 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-04-25T01:47:27.530 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.530 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.530 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.530 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T01:47:27.530 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.530 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.530 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.530 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-04-25T01:47:27.530 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-04-25T01:47:27.530 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.530 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.530 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.531 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T01:47:27.531 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.531 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.531 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.531 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T01:47:27.531 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.531 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.531 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.531 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-04-25T01:47:27.531 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-04-25T01:47:27.531 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.531 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.531 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.532 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.533 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.533 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.533 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.533 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T01:47:27.533 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.533 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.533 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.533 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-04-25T01:47:27.533 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-04-25T01:47:27.533 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.533 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.533 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.533 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T01:47:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-04-25T01:47:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-04-25T01:47:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.535 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.535 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.535 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.535 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T01:47:27.535 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.535 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.535 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.535 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-04-25T01:47:27.535 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-04-25T01:47:27.535 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.535 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.535 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.535 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.536 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.536 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.536 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.536 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T01:47:27.536 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.536 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.536 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.536 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-04-25T01:47:27.536 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-04-25T01:47:27.536 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.536 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.536 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.537 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-04-25T01:47:27.537 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.537 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.537 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.537 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-04-25T01:47:27.537 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.537 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.537 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.537 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-04-25T01:47:27.537 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-04-25T01:47:27.537 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.537 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.537 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.538 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.538 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.538 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.538 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.538 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T01:47:27.538 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.538 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.538 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.538 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-04-25T01:47:27.538 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-04-25T01:47:27.538 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.538 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.538 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.539 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.539 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.539 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.539 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.539 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T01:47:27.539 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.539 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.539 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.539 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-04-25T01:47:27.539 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-04-25T01:47:27.539 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.539 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.540 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.540 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.540 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.540 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.540 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.540 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T01:47:27.540 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.540 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.540 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.540 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-04-25T01:47:27.540 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-04-25T01:47:27.540 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.540 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.541 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T01:47:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-04-25T01:47:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-04-25T01:47:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.542 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.543 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.543 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.543 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.543 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.543 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T01:47:27.543 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.543 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.543 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.543 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.543 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.543 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-04-25T01:47:27.543 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.544 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.544 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.544 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.544 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.544 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.544 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.544 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.544 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.544 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.544 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.544 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.544 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.544 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.544 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.546 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.546 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.546 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.546 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.546 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.546 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.546 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.546 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.546 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.546 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.546 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.546 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.546 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.547 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.547 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.547 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.547 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.547 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.547 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.547 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.547 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.547 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.547 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.547 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.547 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.547 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.547 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.548 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.548 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.548 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.548 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.548 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.548 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.548 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.548 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.548 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.548 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.548 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.548 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.549 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.550 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-04-25T01:47:27.550 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-04-25T01:47:27.550 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.550 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.550 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.550 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-04-25T01:47:27.550 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.550 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.550 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.550 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T01:47:27.550 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.550 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.550 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.551 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.551 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.551 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-04-25T01:47:27.551 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.551 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.551 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.551 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T01:47:27.551 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T01:47:27.551 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.551 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.551 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.551 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.552 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.552 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.552 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.552 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.552 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.552 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.552 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.552 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T01:47:27.552 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T01:47:27.552 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.552 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.552 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.552 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.553 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.553 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.553 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.553 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.553 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.553 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.553 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.553 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T01:47:27.553 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T01:47:27.553 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.553 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.553 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.553 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.554 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.554 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.554 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.554 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.554 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.554 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.554 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.554 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-04-25T01:47:27.554 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T01:47:27.554 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.554 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.554 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.554 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.555 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.555 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.555 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.555 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.555 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.555 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.555 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.555 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T01:47:27.555 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T01:47:27.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.555 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.555 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.555 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.556 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.556 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.556 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.556 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.556 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.556 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.556 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.556 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T01:47:27.556 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T01:47:27.556 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.556 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.556 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.556 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.557 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.557 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.557 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.557 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.557 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.557 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.557 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.557 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T01:47:27.557 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T01:47:27.557 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.557 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.557 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.558 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.558 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.558 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.558 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.558 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.558 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.558 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.558 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.558 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T01:47:27.558 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T01:47:27.558 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.558 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.558 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.559 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T01:47:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T01:47:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.561 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.561 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.561 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.561 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.561 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.561 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.561 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.561 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T01:47:27.561 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T01:47:27.561 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.561 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.561 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.561 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.562 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.563 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.564 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.565 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.565 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.565 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.565 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.565 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T01:47:27.565 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T01:47:27.565 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.565 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.565 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.565 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T01:47:27.565 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.565 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.565 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.566 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.567 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.567 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.567 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.567 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-04-25T01:47:27.567 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-04-25T01:47:27.567 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.567 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.567 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.567 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.567 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.567 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.567 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.568 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.568 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.568 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.568 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.568 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T01:47:27.568 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T01:47:27.568 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.568 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.568 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.568 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T01:47:27.568 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.568 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.568 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.569 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.569 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.569 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.569 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.569 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T01:47:27.569 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T01:47:27.569 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.569 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.569 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.569 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.569 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.569 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.569 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T01:47:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T01:47:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.571 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.571 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.571 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.571 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.571 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T01:47:27.571 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T01:47:27.571 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.571 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.571 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.571 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.571 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.571 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.571 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.572 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.573 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.573 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.573 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.573 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T01:47:27.573 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T01:47:27.573 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.573 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.573 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.573 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.573 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.573 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.573 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.573 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.574 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.574 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.574 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.574 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T01:47:27.574 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T01:47:27.574 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.574 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.574 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.574 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.574 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.574 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.574 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.575 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.575 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.575 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.575 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.575 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T01:47:27.575 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T01:47:27.575 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.575 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.575 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.575 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.575 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.575 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.575 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.576 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.576 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.576 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.576 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.576 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T01:47:27.576 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T01:47:27.576 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.576 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.576 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.576 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.576 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.576 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.576 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T01:47:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T01:47:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.577 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.578 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.578 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.578 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.578 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.578 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T01:47:27.578 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T01:47:27.578 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.578 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.578 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.578 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.578 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.578 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.579 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.579 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.579 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.579 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.579 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.579 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T01:47:27.579 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T01:47:27.579 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.579 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.579 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.579 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.579 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.579 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.580 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.580 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.580 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.580 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.580 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.580 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T01:47:27.580 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T01:47:27.580 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.580 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.580 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.580 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.580 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.580 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.581 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.581 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.581 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.581 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.581 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.581 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T01:47:27.581 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T01:47:27.581 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.581 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.581 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.581 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.581 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.581 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.582 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.582 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.582 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.582 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.582 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.582 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T01:47:27.582 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T01:47:27.582 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.582 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.582 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.582 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.582 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.582 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.583 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.583 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.583 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.583 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.583 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.583 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T01:47:27.583 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T01:47:27.583 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.583 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.583 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.583 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T01:47:27.583 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.584 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.584 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.584 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.584 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.584 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.584 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.584 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T01:47:27.584 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T01:47:27.584 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.584 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.584 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.584 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.584 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.585 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.585 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.585 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.585 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.585 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.585 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.585 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T01:47:27.585 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T01:47:27.585 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.585 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.585 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.585 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-04-25T01:47:27.585 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.586 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.586 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.586 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.586 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.586 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.586 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.586 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T01:47:27.586 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T01:47:27.586 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.586 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.586 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.586 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-04-25T01:47:27.586 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.587 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.587 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.587 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.587 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.587 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.587 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.587 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T01:47:27.587 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T01:47:27.587 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.587 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.587 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.587 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-04-25T01:47:27.588 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.589 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.590 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.590 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.590 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.590 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.590 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.590 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T01:47:27.590 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T01:47:27.590 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.590 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.590 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.590 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.590 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.590 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.591 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.591 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.591 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.591 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.591 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.591 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T01:47:27.591 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T01:47:27.591 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.591 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.591 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.591 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:27.591 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.591 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.592 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.592 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.592 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.592 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.592 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.592 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T01:47:27.592 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T01:47:27.592 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.592 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.592 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.592 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.592 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.592 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.593 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.594 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.594 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.594 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.594 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.594 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-04-25T01:47:27.594 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-04-25T01:47:27.594 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.594 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.594 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.594 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.594 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.594 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.594 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.595 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.596 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.596 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.596 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.596 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-04-25T01:47:27.596 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-04-25T01:47:27.596 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.596 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.596 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.596 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.596 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.596 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.596 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.596 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.597 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.598 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.598 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.598 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-04-25T01:47:27.598 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-04-25T01:47:27.598 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.598 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.598 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.598 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.598 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.598 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.598 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.598 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.598 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.599 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.599 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.599 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.599 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.599 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.599 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.599 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.599 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.599 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.599 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.599 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.599 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.599 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.600 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.600 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.600 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.600 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.600 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.600 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.600 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.600 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.600 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.600 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.600 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.600 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.600 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.601 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.601 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.601 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.601 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.601 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.601 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.601 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.601 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.601 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.601 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.601 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.601 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.601 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.602 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.602 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.602 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.602 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.602 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.602 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.602 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.602 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.602 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.602 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.602 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.602 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.602 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.602 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.603 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.603 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.603 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.603 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.603 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.603 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.603 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.603 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.603 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.603 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.603 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.603 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.603 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.604 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.605 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.605 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-04-25T01:47:27.605 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T01:47:27.605 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.605 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.605 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.605 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.605 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.605 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.605 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.605 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.605 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.605 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.606 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.606 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T01:47:27.606 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T01:47:27.606 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.606 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.606 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.606 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-04-25T01:47:27.606 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.606 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.606 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.606 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.606 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.607 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.607 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.607 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T01:47:27.607 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T01:47:27.607 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.607 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.607 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.607 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-04-25T01:47:27.607 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.607 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.607 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.607 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.608 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.609 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.609 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.609 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-04-25T01:47:27.609 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-04-25T01:47:27.609 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.609 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.609 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.609 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-04-25T01:47:27.609 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.609 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.609 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.609 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.609 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.610 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.610 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.610 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-04-25T01:47:27.610 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-04-25T01:47:27.610 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.610 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.610 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.610 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-04-25T01:47:27.610 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.610 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.610 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.610 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-04-25T01:47:27.611 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-04-25T01:47:27.612 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.612 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.612 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.612 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.612 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.612 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-04-25T01:47:27.612 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-04-25T01:47:27.612 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.612 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.612 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.612 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.612 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.613 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-04-25T01:47:27.614 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-04-25T01:47:27.614 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.614 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.614 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.614 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.614 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.614 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.614 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.614 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-04-25T01:47:27.614 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.614 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.614 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.615 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.616 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.616 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.616 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.616 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.616 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.616 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.616 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.616 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.616 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.616 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.616 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.616 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.617 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.618 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.618 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.618 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.618 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.618 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.618 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.618 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.618 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.618 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.618 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.618 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.618 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.618 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-04-25T01:47:27.619 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-04-25T01:47:27.620 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.621 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.622 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.622 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-04-25T01:47:27.622 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.622 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.622 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.622 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-04-25T01:47:27.622 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.622 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.622 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.622 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-04-25T01:47:27.622 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-04-25T01:47:27.622 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.623 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.623 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.623 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-04-25T01:47:27.623 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.623 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.623 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.623 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T01:47:27.623 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.623 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.623 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.623 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-04-25T01:47:27.623 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-04-25T01:47:27.623 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.624 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.624 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.624 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.624 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.624 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.624 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.624 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T01:47:27.624 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.624 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.624 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.624 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:27.624 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-04-25T01:47:27.625 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.626 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.626 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.626 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-04-25T01:47:27.626 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.626 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.626 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.626 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T01:47:27.626 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.626 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.626 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.626 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.626 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.626 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.626 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.627 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.628 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.629 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.630 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.630 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.630 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.630 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.630 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.630 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.630 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.630 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.630 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.630 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.630 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.630 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.630 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.630 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.632 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-04-25T01:47:27.632 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-04-25T01:47:27.632 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.632 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.632 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.632 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-04-25T01:47:27.632 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.632 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.632 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.632 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.632 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.633 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.633 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.633 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-04-25T01:47:27.633 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-04-25T01:47:27.633 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.633 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.633 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.633 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-04-25T01:47:27.633 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.633 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.633 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.634 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.634 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.634 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.634 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.634 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:27.634 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:27.634 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.634 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.634 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.634 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-04-25T01:47:27.634 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.634 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.634 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.635 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.635 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.635 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.635 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.635 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.635 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.635 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.635 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-04-25T01:47:27.635 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-04-25T01:47:27.635 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-04-25T01:47:27.635 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.635 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-04-25T01:47:27.636 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-04-25T01:47:27.636 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.636 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.636 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.636 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-04-25T01:47:27.636 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.636 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.636 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.636 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T01:47:27.636 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.636 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.636 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.636 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-04-25T01:47:27.637 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-04-25T01:47:27.637 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.637 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.637 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.637 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-04-25T01:47:27.637 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.637 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.637 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.637 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T01:47:27.637 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.637 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.637 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.637 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-04-25T01:47:27.638 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-04-25T01:47:27.638 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.638 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.638 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.638 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.638 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.638 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.638 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.638 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-04-25T01:47:27.638 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.638 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.638 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.639 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:27.639 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:27.639 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.639 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.639 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.639 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-04-25T01:47:27.639 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-04-25T01:47:27.639 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.639 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.639 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T01:47:27.639 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.639 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.639 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.640 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.641 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.642 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.642 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.642 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.642 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.642 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.642 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.642 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.642 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.642 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.642 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.642 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.642 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.642 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.643 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.643 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.643 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.643 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.643 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.643 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.643 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.643 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.643 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.643 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.643 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.643 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.643 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-04-25T01:47:27.644 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-04-25T01:47:27.645 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.646 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.646 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.646 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-04-25T01:47:27.646 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.646 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.646 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.646 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-04-25T01:47:27.646 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.646 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.646 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.646 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-04-25T01:47:27.646 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-04-25T01:47:27.646 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.647 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.647 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.647 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-04-25T01:47:27.647 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.647 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.647 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.647 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T01:47:27.647 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.647 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.647 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.647 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-04-25T01:47:27.647 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-04-25T01:47:27.647 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.648 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.648 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.648 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T01:47:27.648 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T01:47:27.648 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-04-25T01:47:27.648 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.648 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T01:47:27.648 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.648 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.648 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.648 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-04-25T01:47:27.648 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-04-25T01:47:27.648 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.649 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.649 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.649 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.649 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.649 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.649 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.649 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-04-25T01:47:27.649 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.649 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.649 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.649 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-04-25T01:47:27.649 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-04-25T01:47:27.649 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.650 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.650 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.650 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-04-25T01:47:27.650 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.650 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.650 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.650 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T01:47:27.650 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.650 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.650 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.650 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.650 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.650 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.651 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-04-25T01:47:27.651 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.651 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.651 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.651 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.651 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.651 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.651 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.651 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.651 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.651 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.651 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.651 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.651 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.652 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.653 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.653 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.653 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.653 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.653 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.653 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.653 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.653 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.653 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.653 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.653 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.653 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.653 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.654 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.655 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.655 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.655 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.655 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.655 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.655 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.655 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.655 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.655 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.655 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.655 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.655 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.655 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.656 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.656 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.656 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.656 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-04-25T01:47:27.656 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.656 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.656 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.656 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.656 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.656 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.656 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.656 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.657 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.657 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.657 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.657 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.657 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.657 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.657 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.657 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.657 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.657 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.657 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.657 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.657 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.657 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.658 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.659 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.659 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.659 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.659 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.659 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.659 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.659 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.659 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.659 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.659 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.659 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.659 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.659 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.660 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.660 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.660 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.660 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.660 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.660 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.660 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.660 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.660 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.660 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.660 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.660 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.660 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.661 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.662 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-04-25T01:47:27.662 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-04-25T01:47:27.662 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.662 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.662 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.662 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-04-25T01:47:27.662 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.662 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.662 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.662 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T01:47:27.662 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.662 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.662 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.663 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-04-25T01:47:27.663 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-04-25T01:47:27.663 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.663 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.663 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.663 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.663 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-04-25T01:47:27.663 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.663 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.663 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T01:47:27.663 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.663 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.663 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.664 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.665 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.665 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.665 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.665 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.665 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.665 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.665 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.665 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.665 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.665 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.665 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.665 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.665 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.666 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.666 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.666 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.666 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.666 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.666 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.666 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.666 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.666 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.666 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.666 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.666 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.666 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.667 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.667 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.667 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.667 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.667 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.667 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.667 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.667 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.667 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.667 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.667 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.667 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.667 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.668 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.668 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.668 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.668 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.668 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.668 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.668 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.668 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.668 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.668 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.668 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.668 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.668 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.668 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.669 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.669 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-04-25T01:47:27.669 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.669 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.669 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.669 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-04-25T01:47:27.669 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-04-25T01:47:27.669 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.669 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.669 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.669 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-04-25T01:47:27.670 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.670 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.670 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.670 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-04-25T01:47:27.670 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T01:47:27.670 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.670 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.670 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.670 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.670 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.670 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.670 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.671 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.671 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.671 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.671 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.671 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.671 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.671 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.671 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.671 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.671 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.671 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.671 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.671 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.671 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.672 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.672 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.672 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.672 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.672 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.672 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.672 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.672 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.672 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.672 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.672 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.672 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.672 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.673 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.674 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.675 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-04-25T01:47:27.676 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-04-25T01:47:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-04-25T01:47:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-04-25T01:47:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-04-25T01:47:27.677 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T01:47:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-04-25T01:47:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-04-25T01:47:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.678 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.679 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T01:47:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T01:47:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-04-25T01:47:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.680 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.682 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.683 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.684 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.685 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-04-25T01:47:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.686 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.687 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.688 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.689 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.690 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-04-25T01:47:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.691 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.692 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.693 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.694 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.695 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-04-25T01:47:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-04-25T01:47:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-04-25T01:47:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T01:47:27.697 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.698 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.699 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.700 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.701 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.701 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.701 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.701 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.701 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.701 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.701 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.701 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.701 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.701 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.701 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.701 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.701 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.701 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.702 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-04-25T01:47:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-04-25T01:47:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.703 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-04-25T01:47:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-04-25T01:47:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-04-25T01:47:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-04-25T01:47:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-04-25T01:47:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.704 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.705 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.705 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.705 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.705 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.705 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.705 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.705 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.705 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.705 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.705 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.705 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.705 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.705 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.705 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.706 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.706 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.706 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.706 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.706 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.706 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.706 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.706 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.706 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.706 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.706 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.706 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.707 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.707 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.707 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.707 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.707 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.707 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.707 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.707 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.707 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.707 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.707 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.707 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.707 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.708 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.709 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.709 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.709 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.709 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.709 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.709 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.709 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.709 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.709 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.709 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.709 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.709 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.709 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.710 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.710 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.710 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-04-25T01:47:27.710 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.710 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.710 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.710 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.710 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.710 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.710 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.710 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.711 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.711 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.711 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.711 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.711 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.711 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.711 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.711 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.711 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.711 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.711 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.711 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.711 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.711 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.712 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.712 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.712 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.712 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.712 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.712 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.712 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.712 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.712 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.712 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.712 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.712 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.712 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.713 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.714 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.714 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.714 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.714 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.714 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.714 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.714 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.714 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.714 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.714 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.714 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.714 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.714 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.715 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.715 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.715 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.715 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.715 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.715 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.715 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.715 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.715 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.715 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-04-25T01:47:27.715 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.716 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.716 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.716 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.716 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.716 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.716 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.716 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.716 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.716 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.716 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.716 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.716 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.716 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.716 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.717 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.717 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.717 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.717 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.717 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.717 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.717 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.717 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.717 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.717 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.717 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.717 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.717 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.718 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.718 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.718 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.718 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.718 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.718 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.718 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.718 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.718 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.718 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.718 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.718 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.718 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.719 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.719 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.719 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.719 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.719 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.719 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.719 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.719 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.719 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.719 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.719 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.719 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.719 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.719 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.720 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.721 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:27.721 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:27.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.721 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.721 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.721 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:27.721 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.721 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.721 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.721 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.721 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.721 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.722 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.722 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-04-25T01:47:27.722 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-04-25T01:47:27.722 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:27.722 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.722 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.722 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-04-25T01:47:27.722 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-04-25T01:47:27.723 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.723 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.723 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-04-25T01:47:27.723 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T01:47:27.723 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.723 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.723 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.724 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.724 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-04-25T01:47:27.724 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.725 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.725 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.725 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-04-25T01:47:27.725 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-04-25T01:47:27.725 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.725 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.725 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.725 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.726 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.726 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.726 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.726 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T01:47:27.726 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.726 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.726 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.726 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-04-25T01:47:27.726 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-04-25T01:47:27.726 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.726 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.726 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.727 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.727 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.727 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.727 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.727 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.727 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.727 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.727 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.727 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.727 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.727 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.727 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.727 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.728 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.728 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.728 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.728 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.728 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.728 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.728 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.728 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.728 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.728 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.728 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.728 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.728 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.728 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.729 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.729 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.729 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.729 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.729 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.729 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.729 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.729 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.729 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.729 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.729 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.729 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.729 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.730 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.730 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.730 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.730 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.730 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.730 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.730 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.730 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.730 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.730 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.730 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.730 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.730 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.731 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-04-25T01:47:27.732 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.733 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.733 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.733 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-04-25T01:47:27.733 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.733 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.733 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.733 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-04-25T01:47:27.733 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-04-25T01:47:27.733 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.733 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.733 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.733 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-04-25T01:47:27.733 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.734 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-04-25T01:47:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-04-25T01:47:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.735 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-04-25T01:47:27.736 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-04-25T01:47:27.736 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.736 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.736 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.736 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.736 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.736 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.736 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.736 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T01:47:27.736 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T01:47:27.736 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.736 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.736 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.737 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.738 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.738 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.738 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.738 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.738 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.738 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.738 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.738 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.738 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.738 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.738 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.738 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.739 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.739 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.739 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.739 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.739 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.739 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.739 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.739 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.739 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.739 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.739 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.739 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.739 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.740 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.741 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.741 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.741 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.741 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.741 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.741 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.741 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.741 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.741 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.741 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.741 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.741 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-04-25T01:47:27.741 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-04-25T01:47:27.742 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.742 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.742 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.742 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.742 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.742 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.742 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.742 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.742 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.742 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.742 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.742 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T01:47:27.742 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T01:47:27.743 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.743 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.743 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.743 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-04-25T01:47:27.743 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.743 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.743 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.743 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.743 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.743 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.743 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.743 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-04-25T01:47:27.743 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-04-25T01:47:27.744 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:27.744 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.744 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.744 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-04-25T01:47:27.744 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.744 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.744 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.744 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.744 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.744 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.744 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.744 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T01:47:27.744 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T01:47:27.745 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.745 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.745 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.745 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-04-25T01:47:27.745 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.745 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.745 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.745 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T01:47:27.745 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.745 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.745 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.745 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T01:47:27.745 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T01:47:27.746 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.746 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.746 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.746 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-04-25T01:47:27.746 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.746 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.746 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.746 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T01:47:27.746 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.746 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.746 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.746 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-04-25T01:47:27.746 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-04-25T01:47:27.747 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-04-25T01:47:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-04-25T01:47:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-04-25T01:47:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.750 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.750 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-04-25T01:47:27.750 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.750 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.750 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.750 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.750 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.750 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.750 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.750 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.750 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.750 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.751 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.751 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.751 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.751 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.751 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.751 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.751 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.751 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.751 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.751 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.751 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.751 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.751 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.752 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.752 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.752 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.752 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.752 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.752 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.752 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.752 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.752 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.752 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.752 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.752 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.752 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.753 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.753 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.753 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.753 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.753 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.753 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.753 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.753 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.753 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.753 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.753 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.753 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.753 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.753 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.754 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.754 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.754 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.754 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.754 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.754 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.754 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.754 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.754 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.754 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.754 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.754 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.754 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.755 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.755 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.755 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.755 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.755 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.755 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.755 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-04-25T01:47:27.755 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-04-25T01:47:27.755 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.755 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.755 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.755 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-04-25T01:47:27.756 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.756 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.756 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.756 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.756 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.756 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.756 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.756 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-04-25T01:47:27.756 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T01:47:27.756 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.756 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.756 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.756 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.757 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.757 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.757 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.757 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.757 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.757 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.757 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.757 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-04-25T01:47:27.757 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-04-25T01:47:27.757 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.757 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.757 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.757 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.758 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.758 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.758 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.758 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.758 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.758 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.758 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.758 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.758 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.758 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-04-25T01:47:27.758 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.758 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.759 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.759 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-04-25T01:47:27.759 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-04-25T01:47:27.759 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.759 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.759 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.759 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.759 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.759 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.759 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.759 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.759 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.759 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.759 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.761 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.762 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.762 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.762 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.762 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.762 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.762 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.762 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.762 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.762 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.762 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.762 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.762 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.762 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.762 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.763 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-04-25T01:47:27.765 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-04-25T01:47:27.766 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.766 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.766 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.766 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.766 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.766 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.766 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.766 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.766 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.766 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.766 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.766 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-04-25T01:47:27.766 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-04-25T01:47:27.767 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.767 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.767 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.767 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.767 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.767 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.767 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.767 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.767 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.767 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.767 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.767 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.767 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.768 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.768 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-04-25T01:47:27.768 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.768 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.768 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.768 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.768 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.768 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.768 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.768 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.768 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.768 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.768 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.768 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.769 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.770 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.770 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.770 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.770 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.770 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.770 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.770 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.770 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.770 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.770 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.770 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.770 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.770 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.771 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.771 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.771 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.771 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.771 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.771 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.771 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.771 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.771 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.771 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.771 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.771 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.771 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.771 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.772 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.772 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.772 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.772 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.772 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.772 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.772 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.772 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.772 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.772 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.772 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.772 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.772 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.773 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.774 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.774 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.774 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.774 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.774 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.774 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.774 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.774 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.774 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.774 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.774 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.774 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.774 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.775 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.775 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.775 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.775 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.775 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.775 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.775 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.775 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.775 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.775 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.775 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.775 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.775 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.775 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.776 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.777 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.778 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.778 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.778 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.778 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.778 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.778 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.778 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.778 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.778 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.778 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-04-25T01:47:27.778 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-04-25T01:47:27.778 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.778 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.779 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.779 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-04-25T01:47:27.779 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.779 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.779 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.779 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T01:47:27.779 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.779 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.779 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.779 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.779 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.779 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-04-25T01:47:27.780 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.780 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.780 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.780 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.780 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.780 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.780 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.780 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.780 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.780 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.780 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.780 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.780 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.780 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.781 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.782 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.782 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.782 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.782 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.782 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.782 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.782 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.782 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.782 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.782 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.782 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.782 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.782 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.783 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.784 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.785 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-04-25T01:47:27.785 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-04-25T01:47:27.785 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.785 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.785 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.785 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.785 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.785 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.785 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.785 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.785 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.785 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.786 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.786 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-04-25T01:47:27.786 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-04-25T01:47:27.786 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.786 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.786 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.786 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-04-25T01:47:27.786 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.786 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.786 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.786 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.786 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.786 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.787 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.787 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-04-25T01:47:27.787 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-04-25T01:47:27.787 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.787 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.787 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.787 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.787 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.787 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.787 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.787 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.787 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.787 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.788 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.789 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-04-25T01:47:27.789 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-04-25T01:47:27.789 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-04-25T01:47:27.789 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.789 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.789 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.789 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.789 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.789 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.789 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.789 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.789 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.789 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.790 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-04-25T01:47:27.790 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-04-25T01:47:27.790 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.790 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.790 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.790 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.790 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.790 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.790 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.790 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.790 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.790 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.791 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.792 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-04-25T01:47:27.792 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-04-25T01:47:27.792 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.792 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.792 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.792 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.792 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.792 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.792 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.792 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.792 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.792 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.793 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-04-25T01:47:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-04-25T01:47:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.794 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.795 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.796 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.797 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.797 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-04-25T01:47:27.797 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.797 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.797 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.797 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-04-25T01:47:27.797 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-04-25T01:47:27.797 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.797 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.797 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.797 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.797 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.798 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.798 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.798 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T01:47:27.798 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.798 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.798 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.798 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-04-25T01:47:27.798 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-04-25T01:47:27.798 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.798 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.798 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.798 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.798 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.799 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.799 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.799 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-04-25T01:47:27.799 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.799 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.799 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.799 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.799 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.799 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.799 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.799 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.799 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.799 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.800 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.800 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.800 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.800 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.800 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.800 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.800 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.800 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.800 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.800 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.800 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.800 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.800 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.801 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.801 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.801 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.801 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.801 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.801 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.801 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.801 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.801 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.801 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.801 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.801 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.801 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.802 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.802 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.802 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.802 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.802 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.802 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.802 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.802 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.802 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.802 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.802 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.802 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.803 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.804 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.804 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.804 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.804 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.804 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.804 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.804 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.804 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.804 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.804 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.804 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-04-25T01:47:27.804 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.805 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.805 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.805 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.805 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.805 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.805 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.805 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.805 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.805 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.805 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.805 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.805 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.805 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.805 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.806 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.806 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.806 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.806 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.806 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.806 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.806 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.806 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.806 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.806 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.806 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.806 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.807 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.808 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.808 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.808 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.808 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.808 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.808 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.808 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.808 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.808 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.808 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.808 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.808 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.808 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.808 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.809 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.809 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.809 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.809 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.809 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.809 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.809 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.809 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.809 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.809 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.809 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.809 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.809 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.810 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.810 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.810 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.810 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.810 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-04-25T01:47:27.810 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.810 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.810 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.810 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.810 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.810 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.810 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.811 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.812 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.812 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.812 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.812 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.812 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.812 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.812 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.812 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.812 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.812 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.812 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.812 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.812 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.813 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.813 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.813 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.813 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.813 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.813 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.813 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.813 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.813 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.813 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.813 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.813 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.813 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.814 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.814 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.814 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.814 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.814 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.814 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.814 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.814 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.814 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.814 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.814 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.814 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.814 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.815 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.817 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:27.817 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:27.817 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.817 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.817 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.817 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-04-25T01:47:27.817 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.817 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.817 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.817 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.817 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.817 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.817 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.818 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.818 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.818 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.818 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.818 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.818 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.818 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.818 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.818 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.818 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.818 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.818 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.818 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.818 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.819 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.819 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.819 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.819 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.819 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.819 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.819 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.819 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.819 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.819 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.819 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.819 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.820 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.820 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.820 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.820 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.820 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.820 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.820 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.820 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.820 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.820 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.820 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.820 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.820 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.822 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.822 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.822 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.822 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.822 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.822 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.822 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.822 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.822 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.822 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.822 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.822 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.822 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.822 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.823 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.823 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-04-25T01:47:27.823 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.823 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.823 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.823 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-04-25T01:47:27.823 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-04-25T01:47:27.823 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.823 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.823 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.823 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.823 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.824 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.824 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.824 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T01:47:27.824 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.824 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.824 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.824 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-04-25T01:47:27.824 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-04-25T01:47:27.824 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.824 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.824 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.824 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.824 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.825 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.825 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.825 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T01:47:27.825 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.825 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.825 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.825 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-04-25T01:47:27.825 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-04-25T01:47:27.825 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.825 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.825 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.825 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.825 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.826 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.826 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.826 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T01:47:27.826 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.826 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.826 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.826 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-04-25T01:47:27.826 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-04-25T01:47:27.826 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.826 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.826 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.826 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.827 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.827 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.827 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.827 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T01:47:27.827 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.827 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.827 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.827 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-04-25T01:47:27.827 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-04-25T01:47:27.827 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.827 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.827 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.827 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.828 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.829 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.830 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.830 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.830 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.830 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.830 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.830 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-04-25T01:47:27.830 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-04-25T01:47:27.830 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.830 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.830 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.830 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T01:47:27.830 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.830 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.831 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.831 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.831 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.831 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.831 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.831 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-04-25T01:47:27.831 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-04-25T01:47:27.831 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.831 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.831 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.831 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.831 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.831 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.832 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.833 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.834 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.834 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.834 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.834 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-04-25T01:47:27.834 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-04-25T01:47:27.834 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.834 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.834 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.834 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.834 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.834 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.834 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.834 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.835 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.835 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.835 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.835 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-04-25T01:47:27.835 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-04-25T01:47:27.835 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.835 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.835 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.835 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.835 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.835 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.835 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.835 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.836 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.837 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.837 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.837 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.837 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.837 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.837 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.837 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.837 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.837 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.837 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.837 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.837 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.837 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.838 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.838 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.838 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.838 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.838 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.838 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.838 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.838 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.838 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.838 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.838 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.839 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.840 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.840 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.840 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.840 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.840 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.840 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.840 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.840 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.840 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.840 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.840 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.840 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.840 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-04-25T01:47:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-04-25T01:47:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.841 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-04-25T01:47:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-04-25T01:47:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.842 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-04-25T01:47:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-04-25T01:47:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T01:47:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.843 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-04-25T01:47:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.844 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.845 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.846 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.847 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.848 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.848 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.848 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.848 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.848 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.848 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.848 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.848 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.848 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.848 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.848 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.848 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.849 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-04-25T01:47:27.850 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.850 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.850 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.850 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.850 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.850 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.850 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.850 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.850 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.850 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.850 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.850 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.850 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.850 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.851 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.852 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.853 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.853 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.853 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.853 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.853 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.853 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.853 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.853 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.853 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.853 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.853 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.853 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.853 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.853 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.854 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.854 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.854 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.854 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.854 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.854 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.854 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.854 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.854 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.854 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.854 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.854 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.854 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-04-25T01:47:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-04-25T01:47:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-04-25T01:47:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T01:47:27.855 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.856 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-04-25T01:47:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-04-25T01:47:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-04-25T01:47:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T01:47:27.857 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-04-25T01:47:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T01:47:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-04-25T01:47:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.858 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T01:47:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-04-25T01:47:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-04-25T01:47:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-04-25T01:47:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-04-25T01:47:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-04-25T01:47:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-04-25T01:47:27.859 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-04-25T01:47:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-04-25T01:47:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-04-25T01:47:27.860 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-04-25T01:47:27.861 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-04-25T01:47:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-04-25T01:47:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-04-25T01:47:27.862 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-04-25T01:47:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.863 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.864 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-04-25T01:47:27.865 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-04-25T01:47:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-04-25T01:47:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-04-25T01:47:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-04-25T01:47:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-04-25T01:47:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-04-25T01:47:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-04-25T01:47:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-04-25T01:47:27.866 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-04-25T01:47:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-04-25T01:47:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-04-25T01:47:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-04-25T01:47:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.867 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.868 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-04-25T01:47:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-04-25T01:47:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-04-25T01:47:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-04-25T01:47:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-04-25T01:47:27.869 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: "services": { 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: "dashboard": "https://172.21.15.78:8443/" 2024-04-25T01:47:27.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: "always_on_modules": { 2024-04-25T01:47:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: "octopus": [ 2024-04-25T01:47:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-04-25T01:47:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-04-25T01:47:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-04-25T01:47:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-04-25T01:47:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-04-25T01:47:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-04-25T01:47:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-04-25T01:47:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-04-25T01:47:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-04-25T01:47:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-04-25T01:47:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.871 INFO:teuthology.orchestra.run.smithi078.stdout: "pacific": [ 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: "quincy": [ 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-04-25T01:47:27.872 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: "reef": [ 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-04-25T01:47:27.873 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-04-25T01:47:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-04-25T01:47:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-04-25T01:47:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-04-25T01:47:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-04-25T01:47:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-25T01:47:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: "squid": [ 2024-04-25T01:47:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-04-25T01:47:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-04-25T01:47:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-04-25T01:47:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-04-25T01:47:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-04-25T01:47:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-04-25T01:47:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-04-25T01:47:27.874 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: "last_failure_osd_epoch": 3, 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: "active_clients": [ 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1827952621 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.875 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "libcephsqlite", 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2208238897 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-04-25T01:47:27.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-04-25T01:47:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-04-25T01:47:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2232805550 2024-04-25T01:47:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T01:47:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-04-25T01:47:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-04-25T01:47:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-25T01:47:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-04-25T01:47:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-04-25T01:47:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 3723981482 2024-04-25T01:47:27.877 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.878 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T01:47:27.878 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:27.878 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-25T01:47:27.878 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:47:27.879 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:27 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2063114750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:47:27.879 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:27 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2711880251' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T01:47:27.879 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:27 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2063114750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:47:27.879 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:27 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2711880251' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T01:47:27.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.547+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/2711880251 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d200a2b90 msgr2=0x7f5d200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:27.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.547+0000 7f5d3c7d5640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d200a2b90 0x7f5d200a5050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5d24005fd0 tx=0x7f5d24005850 comp rx=0 tx=0).stop 2024-04-25T01:47:27.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.547+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/2711880251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34100700 msgr2=0x7f5d3406a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:27.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.547+0000 7f5d3c7d5640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34100700 0x7f5d3406a950 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000c970 tx=0x7f5d3000ce30 comp rx=0 tx=0).stop 2024-04-25T01:47:27.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.547+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/2711880251 shutdown_connections 2024-04-25T01:47:27.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.547+0000 7f5d3c7d5640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d341022b0 0x7f5d3419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:27.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.547+0000 7f5d3c7d5640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d341018f0 0x7f5d3419c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:27.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.547+0000 7f5d3c7d5640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f5d200a2b90 0x7f5d200a5050 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:27.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.547+0000 7f5d3c7d5640 1 --2- 172.21.15.78:0/2711880251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d34100700 0x7f5d3406a950 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:27.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.547+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/2711880251 >> 172.21.15.78:0/2711880251 conn(0x7f5d340fbc90 msgr2=0x7f5d34104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:27.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.547+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/2711880251 shutdown_connections 2024-04-25T01:47:27.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:47:26.547+0000 7f5d3c7d5640 1 -- 172.21.15.78:0/2711880251 wait complete. 2024-04-25T01:47:27.886 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T01:47:27.887 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T01:47:27.887 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:27 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2063114750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:47:27.887 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:27 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2711880251' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T01:47:28.132 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:47:28.256 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:28 smithi078 ceph-mon[29469]: pgmap v419: 129 pgs: 129 active+clean; 217 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 837 KiB/s wr, 293 op/s 2024-04-25T01:47:28.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:28 smithi152 ceph-mon[32465]: pgmap v419: 129 pgs: 129 active+clean; 217 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 837 KiB/s wr, 293 op/s 2024-04-25T01:47:28.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:28 smithi196 ceph-mon[32978]: pgmap v419: 129 pgs: 129 active+clean; 217 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 837 KiB/s wr, 293 op/s 2024-04-25T01:47:30.085 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:30 smithi078 ceph-mon[29469]: pgmap v420: 129 pgs: 129 active+clean; 205 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 815 KiB/s wr, 278 op/s 2024-04-25T01:47:30.215 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:47:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:47:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 319, 2024-04-25T01:47:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 311, 2024-04-25T01:47:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 817616, 2024-04-25T01:47:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 118204, 2024-04-25T01:47:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-04-25T01:47:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-04-25T01:47:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.037808462, 2024-04-25T01:47:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.012207829, 2024-04-25T01:47:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.018826617, 2024-04-25T01:47:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.007960618, 2024-04-25T01:47:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 311, 2024-04-25T01:47:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.013220165, 2024-04-25T01:47:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000042508 2024-04-25T01:47:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 741680, 2024-04-25T01:47:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 46336 2024-04-25T01:47:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:47:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 167, 2024-04-25T01:47:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 222, 2024-04-25T01:47:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 41648, 2024-04-25T01:47:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 223992, 2024-04-25T01:47:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-04-25T01:47:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 4, 2024-04-25T01:47:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.018558552, 2024-04-25T01:47:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.006668980, 2024-04-25T01:47:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.004782798, 2024-04-25T01:47:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.005539667, 2024-04-25T01:47:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 220, 2024-04-25T01:47:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.008386982, 2024-04-25T01:47:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000038122 2024-04-25T01:47:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:47:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 171136 2024-04-25T01:47:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:47:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 232, 2024-04-25T01:47:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 234, 2024-04-25T01:47:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 62397, 2024-04-25T01:47:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 73777, 2024-04-25T01:47:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-04-25T01:47:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2024-04-25T01:47:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.018108527, 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.005614627, 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.005392983, 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.005438773, 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 232, 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.008118400, 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000034993 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:47:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:47:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-04-25T01:47:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-04-25T01:47:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-04-25T01:47:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-04-25T01:47:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-04-25T01:47:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-04-25T01:47:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1043, 2024-04-25T01:47:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.026432201, 2024-04-25T01:47:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000025342 2024-04-25T01:47:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-04-25T01:47:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-04-25T01:47:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-04-25T01:47:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-04-25T01:47:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 0, 2024-04-25T01:47:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 0, 2024-04-25T01:47:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-04-25T01:47:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-04-25T01:47:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 0, 2024-04-25T01:47:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2024-04-25T01:47:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-04-25T01:47:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 0, 2024-04-25T01:47:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 0, 2024-04-25T01:47:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2024-04-25T01:47:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 0, 2024-04-25T01:47:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 0, 2024-04-25T01:47:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 0, 2024-04-25T01:47:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 0, 2024-04-25T01:47:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 0, 2024-04-25T01:47:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 0, 2024-04-25T01:47:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-04-25T01:47:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-04-25T01:47:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 1, 2024-04-25T01:47:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 0, 2024-04-25T01:47:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 0, 2024-04-25T01:47:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 0, 2024-04-25T01:47:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 0, 2024-04-25T01:47:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:47:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-04-25T01:47:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2024-04-25T01:47:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-04-25T01:47:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-04-25T01:47:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 0, 2024-04-25T01:47:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:47:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:47:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 0, 2024-04-25T01:47:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 0, 2024-04-25T01:47:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T01:47:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 0, 2024-04-25T01:47:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:47:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:47:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:47:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 0, 2024-04-25T01:47:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 0, 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 0, 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 0 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 0, 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 0, 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 0, 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 0, 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 0, 2024-04-25T01:47:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-04-25T01:47:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-04-25T01:47:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-04-25T01:47:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:47:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:47:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-04-25T01:47:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-04-25T01:47:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:47:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:47:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:47:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 0, 2024-04-25T01:47:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-04-25T01:47:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:47:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-04-25T01:47:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-04-25T01:47:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-04-25T01:47:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-04-25T01:47:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 0, 2024-04-25T01:47:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-04-25T01:47:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-04-25T01:47:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 6, 2024-04-25T01:47:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-04-25T01:47:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-04-25T01:47:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 0, 2024-04-25T01:47:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-04-25T01:47:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-04-25T01:47:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 2, 2024-04-25T01:47:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 2, 2024-04-25T01:47:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-04-25T01:47:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-04-25T01:47:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 4194304, 2024-04-25T01:47:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 4195956, 2024-04-25T01:47:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 4195815, 2024-04-25T01:47:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-04-25T01:47:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 6 2024-04-25T01:47:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-04-25T01:47:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 3, 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 3, 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 0, 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 1, 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 1, 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 0, 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 0, 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 0, 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 0, 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 0, 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 0, 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 49904, 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 215420 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 0, 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 0, 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 0, 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 0, 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 0, 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-04-25T01:47:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-04-25T01:47:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-04-25T01:47:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-04-25T01:47:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-04-25T01:47:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-04-25T01:47:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-04-25T01:47:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-04-25T01:47:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-04-25T01:47:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-04-25T01:47:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-04-25T01:47:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-04-25T01:47:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-04-25T01:47:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-04-25T01:47:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-04-25T01:47:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-04-25T01:47:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-04-25T01:47:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-04-25T01:47:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-04-25T01:47:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-04-25T01:47:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-04-25T01:47:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-04-25T01:47:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-04-25T01:47:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-04-25T01:47:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:47:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:47:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-04-25T01:47:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-04-25T01:47:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 0, 2024-04-25T01:47:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 2, 2024-04-25T01:47:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 2, 2024-04-25T01:47:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 1, 2024-04-25T01:47:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-04-25T01:47:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 0, 2024-04-25T01:47:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 0, 2024-04-25T01:47:30.247 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 1247, 2024-04-25T01:47:30.247 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-04-25T01:47:30.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.247 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-04-25T01:47:30.247 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:47:30.247 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-04-25T01:47:30.247 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:47:30.247 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:47:30.247 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:47:30.247 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:47:30.247 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:47:30.247 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:47:30.247 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:47:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 189256, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 49, 2024-04-25T01:47:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:47:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-04-25T01:47:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-04-25T01:47:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-04-25T01:47:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:47:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-04-25T01:47:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:47:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-04-25T01:47:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 9096, 2024-04-25T01:47:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 176, 2024-04-25T01:47:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:47:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:47:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-04-25T01:47:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-04-25T01:47:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 8464, 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 13, 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 624, 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 21, 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 3, 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 624, 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-04-25T01:47:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 624, 2024-04-25T01:47:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.370994724, 2024-04-25T01:47:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000594542 2024-04-25T01:47:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-04-25T01:47:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-04-25T01:47:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 624, 2024-04-25T01:47:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 625 2024-04-25T01:47:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 624, 2024-04-25T01:47:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 624, 2024-04-25T01:47:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 0, 2024-04-25T01:47:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-04-25T01:47:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-04-25T01:47:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 389, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 0, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 233, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 0, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 0, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 0, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 0, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 1, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 0, 2024-04-25T01:47:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:47:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-04-25T01:47:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:47:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-04-25T01:47:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-04-25T01:47:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-04-25T01:47:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:47:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-04-25T01:47:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:47:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-04-25T01:47:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-04-25T01:47:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-04-25T01:47:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-04-25T01:47:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-04-25T01:47:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-04-25T01:47:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-04-25T01:47:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-04-25T01:47:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-04-25T01:47:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-04-25T01:47:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-04-25T01:47:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-04-25T01:47:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-04-25T01:47:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-04-25T01:47:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-04-25T01:47:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-04-25T01:47:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-04-25T01:47:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 94, 2024-04-25T01:47:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-04-25T01:47:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 4, 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 5, 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 5, 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 0, 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 2, 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 0 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 0, 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-04-25T01:47:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2024-04-25T01:47:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-04-25T01:47:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-04-25T01:47:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T01:47:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-04-25T01:47:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 0, 2024-04-25T01:47:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 0, 2024-04-25T01:47:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 0, 2024-04-25T01:47:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-04-25T01:47:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:47:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T01:47:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-25T01:47:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T01:47:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 714, 2024-04-25T01:47:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 849844, 2024-04-25T01:47:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 714, 2024-04-25T01:47:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-25T01:47:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-25T01:47:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 714, 2024-04-25T01:47:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 849844, 2024-04-25T01:47:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T01:47:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-04-25T01:47:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T01:47:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-25T01:47:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T01:47:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-25T01:47:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-25T01:47:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 624, 2024-04-25T01:47:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 1657, 2024-04-25T01:47:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 624, 2024-04-25T01:47:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 1657, 2024-04-25T01:47:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 624, 2024-04-25T01:47:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 624, 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 624, 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 624, 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-25T01:47:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-25T01:47:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-25T01:47:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-04-25T01:47:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-04-25T01:47:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T01:47:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x56407de046e0": { 2024-04-25T01:47:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T01:47:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-04-25T01:47:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T01:47:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-25T01:47:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-25T01:47:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-25T01:47:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-25T01:47:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-04-25T01:47:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-04-25T01:47:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T01:47:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:30.264 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:47:30.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:30 smithi152 ceph-mon[32465]: pgmap v420: 129 pgs: 129 active+clean; 205 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 815 KiB/s wr, 278 op/s 2024-04-25T01:47:30.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:30 smithi196 ceph-mon[32978]: pgmap v420: 129 pgs: 129 active+clean; 205 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 71 MiB/s rd, 815 KiB/s wr, 278 op/s 2024-04-25T01:47:30.980 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-25T01:47:30.980 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-25T01:47:30.981 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-25T01:47:30.981 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-04-25T01:47:30.981 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T01:47:30.982 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T01:47:31.228 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:47:32.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:32 smithi196 ceph-mon[32978]: pgmap v421: 129 pgs: 129 active+clean; 49 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 803 KiB/s wr, 275 op/s 2024-04-25T01:47:32.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:32 smithi078 ceph-mon[29469]: pgmap v421: 129 pgs: 129 active+clean; 49 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 803 KiB/s wr, 275 op/s 2024-04-25T01:47:32.552 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:32 smithi152 ceph-mon[32465]: pgmap v421: 129 pgs: 129 active+clean; 49 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 803 KiB/s wr, 275 op/s 2024-04-25T01:47:32.967 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:47:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:47:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 4, 2024-04-25T01:47:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 4, 2024-04-25T01:47:32.967 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 3072, 2024-04-25T01:47:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 448, 2024-04-25T01:47:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-04-25T01:47:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-04-25T01:47:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.001293948, 2024-04-25T01:47:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000033542, 2024-04-25T01:47:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000466570, 2024-04-25T01:47:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:47:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3, 2024-04-25T01:47:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000663846, 2024-04-25T01:47:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000221282 2024-04-25T01:47:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:32.968 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-25T01:47:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T01:47:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:47:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 85, 2024-04-25T01:47:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 80, 2024-04-25T01:47:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 715152, 2024-04-25T01:47:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 48192, 2024-04-25T01:47:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-04-25T01:47:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-04-25T01:47:32.969 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.016016702, 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.004913838, 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.014279294, 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.001418092, 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 79, 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.003322271, 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000042054 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 715152, 2024-04-25T01:47:32.970 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 48192 2024-04-25T01:47:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:47:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1, 2024-04-25T01:47:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 60, 2024-04-25T01:47:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 160, 2024-04-25T01:47:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 88224, 2024-04-25T01:47:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 3, 2024-04-25T01:47:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-04-25T01:47:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.004893527, 2024-04-25T01:47:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.003175367, 2024-04-25T01:47:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001183716, 2024-04-25T01:47:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:47:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:32.971 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 58, 2024-04-25T01:47:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.002729441, 2024-04-25T01:47:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000047059 2024-04-25T01:47:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:47:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 88224 2024-04-25T01:47:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-04-25T01:47:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:47:32.973 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:47:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:47:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:47:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:47:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-04-25T01:47:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:47:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-04-25T01:47:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:47:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:47:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:47:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:47:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:47:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:47:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:47:32.974 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-04-25T01:47:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:47:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:47:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:47:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-04-25T01:47:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-04-25T01:47:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-04-25T01:47:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:47:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:47:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:47:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:47:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148863, 2024-04-25T01:47:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 28, 2024-04-25T01:47:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:47:32.975 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-04-25T01:47:32.976 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-04-25T01:47:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-04-25T01:47:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-04-25T01:47:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-04-25T01:47:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:47:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T01:47:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-25T01:47:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T01:47:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 86, 2024-04-25T01:47:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 703209, 2024-04-25T01:47:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 86, 2024-04-25T01:47:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-25T01:47:32.977 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-25T01:47:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 86, 2024-04-25T01:47:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 703209, 2024-04-25T01:47:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T01:47:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:32.978 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:32.978 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:47:33.791 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T01:47:33.791 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T01:47:33.791 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T01:47:33.791 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T01:47:33.791 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T01:47:33.791 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T01:47:34.088 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:47:34.355 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:34 smithi078 ceph-mon[29469]: pgmap v422: 129 pgs: 129 active+clean; 17 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 401 KiB/s wr, 164 op/s 2024-04-25T01:47:34.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:34 smithi196 ceph-mon[32978]: pgmap v422: 129 pgs: 129 active+clean; 17 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 401 KiB/s wr, 164 op/s 2024-04-25T01:47:34.552 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:34 smithi152 ceph-mon[32465]: pgmap v422: 129 pgs: 129 active+clean; 17 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 401 KiB/s wr, 164 op/s 2024-04-25T01:47:35.852 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:47:35.852 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:47:35.852 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 327, 2024-04-25T01:47:35.853 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 318, 2024-04-25T01:47:35.853 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 820250, 2024-04-25T01:47:35.853 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 121074, 2024-04-25T01:47:35.853 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-04-25T01:47:35.853 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-04-25T01:47:35.853 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.039901347, 2024-04-25T01:47:35.853 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.015049509, 2024-04-25T01:47:35.853 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.019330852, 2024-04-25T01:47:35.853 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.007520570, 2024-04-25T01:47:35.853 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:35.853 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 318, 2024-04-25T01:47:35.853 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.014698401, 2024-04-25T01:47:35.854 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000046221 2024-04-25T01:47:35.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.854 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:35.854 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.854 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.854 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.854 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 742224, 2024-04-25T01:47:35.854 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 47616 2024-04-25T01:47:35.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.854 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:47:35.854 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 353, 2024-04-25T01:47:35.855 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 409, 2024-04-25T01:47:35.855 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 91632, 2024-04-25T01:47:35.855 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 283305, 2024-04-25T01:47:35.855 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-04-25T01:47:35.855 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 4, 2024-04-25T01:47:35.855 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.025203246, 2024-04-25T01:47:35.855 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.008714238, 2024-04-25T01:47:35.855 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.007176829, 2024-04-25T01:47:35.855 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.007324744, 2024-04-25T01:47:35.855 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:35.855 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 407, 2024-04-25T01:47:35.855 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.011369499, 2024-04-25T01:47:35.855 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000027934 2024-04-25T01:47:35.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.856 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:35.856 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.856 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.856 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.856 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:47:35.856 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 171312 2024-04-25T01:47:35.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.856 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:47:35.856 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 4, 2024-04-25T01:47:35.856 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 6, 2024-04-25T01:47:35.856 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 1629, 2024-04-25T01:47:35.856 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 1284, 2024-04-25T01:47:35.857 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-04-25T01:47:35.857 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 4, 2024-04-25T01:47:35.857 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.001223446, 2024-04-25T01:47:35.857 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000072618, 2024-04-25T01:47:35.857 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000962095, 2024-04-25T01:47:35.857 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000087464, 2024-04-25T01:47:35.857 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:35.857 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 4, 2024-04-25T01:47:35.857 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.002000002, 2024-04-25T01:47:35.857 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000500000 2024-04-25T01:47:35.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.857 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:35.857 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.858 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.858 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.858 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:47:35.858 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:47:35.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.858 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-04-25T01:47:35.858 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-04-25T01:47:35.858 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-04-25T01:47:35.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1046, 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.022778852, 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000021777 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.859 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:35.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.860 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-04-25T01:47:35.860 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 0, 2024-04-25T01:47:35.860 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 0, 2024-04-25T01:47:35.860 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-04-25T01:47:35.860 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.860 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.860 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.860 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-04-25T01:47:35.860 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 0, 2024-04-25T01:47:35.860 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2024-04-25T01:47:35.860 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-04-25T01:47:35.861 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 0, 2024-04-25T01:47:35.861 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 0, 2024-04-25T01:47:35.861 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2024-04-25T01:47:35.861 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 0, 2024-04-25T01:47:35.861 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 0, 2024-04-25T01:47:35.861 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 0, 2024-04-25T01:47:35.861 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 0, 2024-04-25T01:47:35.861 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 0, 2024-04-25T01:47:35.861 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 0, 2024-04-25T01:47:35.861 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-04-25T01:47:35.861 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-04-25T01:47:35.861 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 1, 2024-04-25T01:47:35.861 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 0, 2024-04-25T01:47:35.862 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 0, 2024-04-25T01:47:35.862 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 0, 2024-04-25T01:47:35.862 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 0, 2024-04-25T01:47:35.862 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:47:35.862 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-04-25T01:47:35.862 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2024-04-25T01:47:35.862 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-04-25T01:47:35.862 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-04-25T01:47:35.862 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 0, 2024-04-25T01:47:35.862 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 0, 2024-04-25T01:47:35.862 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 0, 2024-04-25T01:47:35.862 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 0, 2024-04-25T01:47:35.862 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-04-25T01:47:35.863 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:47:35.863 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-04-25T01:47:35.863 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 0, 2024-04-25T01:47:35.863 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 0, 2024-04-25T01:47:35.863 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2024-04-25T01:47:35.863 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:47:35.863 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-04-25T01:47:35.863 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:47:35.863 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:47:35.863 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:47:35.863 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:47:35.863 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:47:35.863 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 0, 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 0, 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 0, 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 0, 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 0, 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 0, 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 0 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2024-04-25T01:47:35.864 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2024-04-25T01:47:35.865 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 0, 2024-04-25T01:47:35.865 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 0, 2024-04-25T01:47:35.865 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 0, 2024-04-25T01:47:35.865 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-04-25T01:47:35.865 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:47:35.865 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 0, 2024-04-25T01:47:35.865 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 0, 2024-04-25T01:47:35.865 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-04-25T01:47:35.865 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-04-25T01:47:35.865 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-04-25T01:47:35.865 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:47:35.865 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:47:35.865 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-04-25T01:47:35.866 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-04-25T01:47:35.866 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:47:35.866 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:47:35.866 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:47:35.866 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 0, 2024-04-25T01:47:35.866 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-04-25T01:47:35.866 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:47:35.866 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-04-25T01:47:35.866 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-04-25T01:47:35.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.866 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-04-25T01:47:35.866 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-04-25T01:47:35.866 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 0, 2024-04-25T01:47:35.866 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 4, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 0, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 2, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 2, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 4194304, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 4195424, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 4195283, 2024-04-25T01:47:35.867 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-04-25T01:47:35.868 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.868 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.868 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.868 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 4 2024-04-25T01:47:35.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.868 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-04-25T01:47:35.868 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 3, 2024-04-25T01:47:35.868 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 3, 2024-04-25T01:47:35.868 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 0, 2024-04-25T01:47:35.868 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 1, 2024-04-25T01:47:35.868 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 1, 2024-04-25T01:47:35.868 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 0, 2024-04-25T01:47:35.868 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 0, 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 0, 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 0, 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 0, 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 0, 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 49112, 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 215420 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 0, 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 0, 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 0, 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 0, 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 0, 2024-04-25T01:47:35.869 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-04-25T01:47:35.870 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.870 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.870 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.870 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-04-25T01:47:35.870 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.870 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.870 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.870 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-04-25T01:47:35.870 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.870 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.870 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.872 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-04-25T01:47:35.872 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.872 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.872 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.872 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-04-25T01:47:35.872 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.872 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.872 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.872 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-04-25T01:47:35.872 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.872 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.872 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.873 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-04-25T01:47:35.873 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.873 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.873 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.873 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-04-25T01:47:35.873 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.873 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.873 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.873 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-04-25T01:47:35.873 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.873 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.874 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.875 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-04-25T01:47:35.875 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.875 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.875 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.875 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-04-25T01:47:35.875 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.875 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.875 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.875 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-04-25T01:47:35.875 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.875 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.876 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.876 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-04-25T01:47:35.876 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.876 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.876 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.876 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-04-25T01:47:35.876 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.876 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.876 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.876 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-04-25T01:47:35.877 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.877 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.877 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.877 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-04-25T01:47:35.877 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.877 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.877 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.877 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-04-25T01:47:35.877 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.877 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.877 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.878 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-04-25T01:47:35.878 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.878 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.878 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.878 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-04-25T01:47:35.878 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.878 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.878 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.878 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-04-25T01:47:35.879 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.879 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.879 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.879 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-04-25T01:47:35.879 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.879 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.879 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.879 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-04-25T01:47:35.879 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.879 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.879 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.880 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-04-25T01:47:35.880 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.880 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.880 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.880 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-04-25T01:47:35.880 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.880 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.880 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.880 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-04-25T01:47:35.880 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.880 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.881 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.881 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-04-25T01:47:35.881 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.881 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.881 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.881 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:47:35.881 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:47:35.881 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-04-25T01:47:35.881 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.881 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.882 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.882 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:35.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.882 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-04-25T01:47:35.882 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 0, 2024-04-25T01:47:35.882 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 1, 2024-04-25T01:47:35.882 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 1, 2024-04-25T01:47:35.882 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 1, 2024-04-25T01:47:35.882 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-04-25T01:47:35.882 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 0, 2024-04-25T01:47:35.882 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 0, 2024-04-25T01:47:35.882 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 1249, 2024-04-25T01:47:35.883 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-04-25T01:47:35.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.883 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-04-25T01:47:35.883 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:47:35.883 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-04-25T01:47:35.883 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:47:35.883 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:47:35.883 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:47:35.883 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:47:35.883 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:47:35.883 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:47:35.883 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:47:35.883 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:47:35.883 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:47:35.884 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:47:35.884 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:47:35.884 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:47:35.884 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:47:35.884 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-04-25T01:47:35.884 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:47:35.884 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-04-25T01:47:35.884 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:47:35.884 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:47:35.884 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:47:35.884 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:47:35.884 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:47:35.885 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:47:35.885 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:47:35.885 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-04-25T01:47:35.885 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:47:35.885 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:47:35.885 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:47:35.885 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-04-25T01:47:35.885 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-04-25T01:47:35.885 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-04-25T01:47:35.885 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:47:35.885 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:47:35.885 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:47:35.886 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:47:35.886 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 197337, 2024-04-25T01:47:35.886 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 53, 2024-04-25T01:47:35.886 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:47:35.886 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-04-25T01:47:35.886 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-04-25T01:47:35.886 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-04-25T01:47:35.886 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:47:35.886 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-04-25T01:47:35.886 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:47:35.886 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-04-25T01:47:35.886 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 9096, 2024-04-25T01:47:35.887 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 176, 2024-04-25T01:47:35.887 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:47:35.887 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:47:35.887 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-04-25T01:47:35.887 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-04-25T01:47:35.887 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 8464, 2024-04-25T01:47:35.887 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 13, 2024-04-25T01:47:35.887 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-04-25T01:47:35.887 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-04-25T01:47:35.887 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-04-25T01:47:35.887 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-04-25T01:47:35.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.887 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-04-25T01:47:35.888 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-04-25T01:47:35.888 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-04-25T01:47:35.888 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 587, 2024-04-25T01:47:35.888 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 21, 2024-04-25T01:47:35.888 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 3, 2024-04-25T01:47:35.888 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 587, 2024-04-25T01:47:35.888 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-04-25T01:47:35.888 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 587, 2024-04-25T01:47:35.888 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 1.179983216, 2024-04-25T01:47:35.888 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.002010192 2024-04-25T01:47:35.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.888 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-04-25T01:47:35.888 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-04-25T01:47:35.888 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 587, 2024-04-25T01:47:35.889 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 588 2024-04-25T01:47:35.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.889 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 587, 2024-04-25T01:47:35.889 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 587, 2024-04-25T01:47:35.889 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 0, 2024-04-25T01:47:35.889 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-04-25T01:47:35.889 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-04-25T01:47:35.889 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 348, 2024-04-25T01:47:35.889 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 0, 2024-04-25T01:47:35.889 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 237, 2024-04-25T01:47:35.889 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 0, 2024-04-25T01:47:35.889 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 0, 2024-04-25T01:47:35.889 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-04-25T01:47:35.889 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-04-25T01:47:35.890 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-04-25T01:47:35.890 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 0, 2024-04-25T01:47:35.890 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 0, 2024-04-25T01:47:35.890 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-04-25T01:47:35.890 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-04-25T01:47:35.890 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-04-25T01:47:35.890 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 1, 2024-04-25T01:47:35.890 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 0, 2024-04-25T01:47:35.890 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:47:35.890 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-04-25T01:47:35.890 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:47:35.890 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-04-25T01:47:35.890 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-04-25T01:47:35.891 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-04-25T01:47:35.892 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-04-25T01:47:35.892 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-04-25T01:47:35.892 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-04-25T01:47:35.892 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-04-25T01:47:35.892 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-04-25T01:47:35.892 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-04-25T01:47:35.892 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 94, 2024-04-25T01:47:35.892 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-04-25T01:47:35.892 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 4, 2024-04-25T01:47:35.892 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 6, 2024-04-25T01:47:35.892 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 6, 2024-04-25T01:47:35.892 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 0, 2024-04-25T01:47:35.892 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-04-25T01:47:35.892 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 0, 2024-04-25T01:47:35.893 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 2, 2024-04-25T01:47:35.893 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 0 2024-04-25T01:47:35.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.893 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-04-25T01:47:35.893 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 0, 2024-04-25T01:47:35.893 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:47:35.893 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-04-25T01:47:35.893 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-04-25T01:47:35.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.893 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-04-25T01:47:35.893 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2024-04-25T01:47:35.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.894 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-04-25T01:47:35.894 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-04-25T01:47:35.894 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T01:47:35.894 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-04-25T01:47:35.894 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 0, 2024-04-25T01:47:35.894 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 0, 2024-04-25T01:47:35.894 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 0, 2024-04-25T01:47:35.894 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-04-25T01:47:35.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.894 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:47:35.894 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T01:47:35.894 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-25T01:47:35.894 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T01:47:35.894 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 680, 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 844650, 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 680, 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 680, 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 844650, 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-04-25T01:47:35.895 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T01:47:35.896 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-25T01:47:35.896 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T01:47:35.896 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-25T01:47:35.896 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-25T01:47:35.896 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:35.896 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:35.896 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 587, 2024-04-25T01:47:35.896 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 1125, 2024-04-25T01:47:35.896 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 587, 2024-04-25T01:47:35.896 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 1125, 2024-04-25T01:47:35.896 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T01:47:35.896 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.896 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.896 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.897 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:35.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.897 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-04-25T01:47:35.897 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T01:47:35.897 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-04-25T01:47:35.897 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T01:47:35.897 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-25T01:47:35.897 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-25T01:47:35.897 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:35.897 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:35.897 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 587, 2024-04-25T01:47:35.897 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 587, 2024-04-25T01:47:35.898 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 587, 2024-04-25T01:47:35.898 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 587, 2024-04-25T01:47:35.898 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T01:47:35.898 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.898 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.898 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.898 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:35.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.898 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:47:35.898 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T01:47:35.898 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-04-25T01:47:35.898 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T01:47:35.898 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-25T01:47:35.899 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-25T01:47:35.899 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:35.899 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:35.899 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-25T01:47:35.899 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-25T01:47:35.899 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-04-25T01:47:35.899 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-04-25T01:47:35.899 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T01:47:35.899 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.899 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.899 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.899 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:35.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:35.899 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x556bc618e6e0": { 2024-04-25T01:47:35.900 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T01:47:35.900 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-04-25T01:47:35.900 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T01:47:35.900 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-04-25T01:47:35.900 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-04-25T01:47:35.900 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:35.900 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:35.900 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-25T01:47:35.900 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-25T01:47:35.900 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-04-25T01:47:35.900 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-04-25T01:47:35.900 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T01:47:35.900 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:35.900 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:35.901 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:35.901 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:35.901 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:35.901 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:47:36.260 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:36 smithi078 ceph-mon[29469]: pgmap v423: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 374 KiB/s wr, 149 op/s 2024-04-25T01:47:36.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:36 smithi196 ceph-mon[32978]: pgmap v423: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 374 KiB/s wr, 149 op/s 2024-04-25T01:47:36.552 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:36 smithi152 ceph-mon[32465]: pgmap v423: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 374 KiB/s wr, 149 op/s 2024-04-25T01:47:36.661 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-25T01:47:36.661 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-25T01:47:36.661 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-25T01:47:36.661 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-04-25T01:47:36.661 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T01:47:36.662 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T01:47:36.899 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T01:47:38.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:38 smithi196 ceph-mon[32978]: pgmap v424: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 286 KiB/s wr, 86 op/s 2024-04-25T01:47:38.391 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:38 smithi078 ceph-mon[29469]: pgmap v424: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 286 KiB/s wr, 86 op/s 2024-04-25T01:47:38.552 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:38 smithi152 ceph-mon[32465]: pgmap v424: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 286 KiB/s wr, 86 op/s 2024-04-25T01:47:38.651 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-25T01:47:38.651 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:47:38.651 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 0, 2024-04-25T01:47:38.651 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2, 2024-04-25T01:47:38.651 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 0, 2024-04-25T01:47:38.651 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 0, 2024-04-25T01:47:38.651 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-04-25T01:47:38.651 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-04-25T01:47:38.651 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.001136450, 2024-04-25T01:47:38.652 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T01:47:38.652 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000309185, 2024-04-25T01:47:38.652 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:47:38.652 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:38.652 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:38.652 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:38.652 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:38.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:38.652 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:38.652 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:38.652 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:38.652 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:38.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:38.653 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:47:38.653 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:47:38.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:38.653 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:47:38.653 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1, 2024-04-25T01:47:38.653 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 62, 2024-04-25T01:47:38.653 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 160, 2024-04-25T01:47:38.653 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 88576, 2024-04-25T01:47:38.653 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 3, 2024-04-25T01:47:38.653 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-04-25T01:47:38.653 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.005654434, 2024-04-25T01:47:38.653 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.003492403, 2024-04-25T01:47:38.653 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001542121, 2024-04-25T01:47:38.653 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T01:47:38.654 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:38.654 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 60, 2024-04-25T01:47:38.654 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.002910003, 2024-04-25T01:47:38.654 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000048500 2024-04-25T01:47:38.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:38.654 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:38.654 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:38.654 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:38.654 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:38.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:38.654 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:47:38.654 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 88576 2024-04-25T01:47:38.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:38.654 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:47:38.655 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 91, 2024-04-25T01:47:38.655 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 84, 2024-04-25T01:47:38.655 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 718768, 2024-04-25T01:47:38.655 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 49920, 2024-04-25T01:47:38.655 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-04-25T01:47:38.655 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-04-25T01:47:38.655 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.016356774, 2024-04-25T01:47:38.655 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.004597145, 2024-04-25T01:47:38.655 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.015138281, 2024-04-25T01:47:38.655 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.001437410, 2024-04-25T01:47:38.655 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:38.655 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 84, 2024-04-25T01:47:38.655 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.004728605, 2024-04-25T01:47:38.655 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000056292 2024-04-25T01:47:38.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:38.656 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:38.656 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:38.656 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:38.656 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:38.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:38.656 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 718768, 2024-04-25T01:47:38.656 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 49920 2024-04-25T01:47:38.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:38.656 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-04-25T01:47:38.656 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-04-25T01:47:38.656 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-04-25T01:47:38.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:38.656 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-04-25T01:47:38.657 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:47:38.657 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-04-25T01:47:38.657 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:47:38.657 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:47:38.657 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:47:38.657 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:47:38.657 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:47:38.657 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:47:38.657 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:47:38.657 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:47:38.657 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:47:38.657 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:47:38.657 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:47:38.657 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:47:38.658 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:47:38.658 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-04-25T01:47:38.658 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:47:38.658 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-04-25T01:47:38.658 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:47:38.658 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:47:38.658 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:47:38.658 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:47:38.658 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:47:38.658 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:47:38.658 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:47:38.658 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-04-25T01:47:38.658 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:47:38.659 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:47:38.659 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:47:38.659 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-04-25T01:47:38.659 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-04-25T01:47:38.659 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-04-25T01:47:38.659 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:47:38.659 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:47:38.659 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:47:38.659 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:47:38.659 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148840, 2024-04-25T01:47:38.659 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 28, 2024-04-25T01:47:38.659 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:47:38.659 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-04-25T01:47:38.659 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-04-25T01:47:38.660 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-04-25T01:47:38.661 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-04-25T01:47:38.661 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-04-25T01:47:38.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-25T01:47:38.661 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:47:38.661 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-04-25T01:47:38.661 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-04-25T01:47:38.661 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-04-25T01:47:38.661 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 88, 2024-04-25T01:47:38.661 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 703469, 2024-04-25T01:47:38.661 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:38.662 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 88, 2024-04-25T01:47:38.662 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-04-25T01:47:38.662 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-04-25T01:47:38.662 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 88, 2024-04-25T01:47:38.662 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 703469, 2024-04-25T01:47:38.662 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-04-25T01:47:38.662 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-04-25T01:47:38.662 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-04-25T01:47:38.662 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-04-25T01:47:38.662 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:38.662 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-25T01:47:38.662 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-25T01:47:39.350 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T01:47:39.351 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T01:47:39.351 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T01:47:39.351 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T01:47:39.351 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T01:47:39.351 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T01:47:39.609 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:47:40.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:40 smithi196 ceph-mon[32978]: pgmap v425: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 32 KiB/s wr, 34 op/s 2024-04-25T01:47:40.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:40 smithi078 ceph-mon[29469]: pgmap v425: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 32 KiB/s wr, 34 op/s 2024-04-25T01:47:40.552 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:40 smithi152 ceph-mon[32465]: pgmap v425: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 32 KiB/s wr, 34 op/s 2024-04-25T01:47:41.570 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-25T01:47:41.570 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:47:41.570 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 4138, 2024-04-25T01:47:41.570 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 4643, 2024-04-25T01:47:41.570 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 1038977, 2024-04-25T01:47:41.570 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 9883113, 2024-04-25T01:47:41.570 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 11, 2024-04-25T01:47:41.570 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-04-25T01:47:41.570 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.279255151, 2024-04-25T01:47:41.570 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.095712272, 2024-04-25T01:47:41.571 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.083360122, 2024-04-25T01:47:41.571 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.083572798, 2024-04-25T01:47:41.571 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:41.571 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 4641, 2024-04-25T01:47:41.571 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.200183799, 2024-04-25T01:47:41.571 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000043133 2024-04-25T01:47:41.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.571 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:41.571 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2175, 2024-04-25T01:47:41.571 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000433932, 2024-04-25T01:47:41.571 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000199 2024-04-25T01:47:41.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.571 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:47:41.572 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:47:41.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.572 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:47:41.572 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 3205, 2024-04-25T01:47:41.572 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 3513, 2024-04-25T01:47:41.572 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 940758, 2024-04-25T01:47:41.572 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 5606944, 2024-04-25T01:47:41.572 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 10, 2024-04-25T01:47:41.572 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 6, 2024-04-25T01:47:41.572 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.223684826, 2024-04-25T01:47:41.572 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.078716668, 2024-04-25T01:47:41.572 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.063604705, 2024-04-25T01:47:41.572 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.066359678, 2024-04-25T01:47:41.573 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:41.573 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 3511, 2024-04-25T01:47:41.573 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.615801448, 2024-04-25T01:47:41.573 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000175392 2024-04-25T01:47:41.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.573 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:41.573 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1094, 2024-04-25T01:47:41.573 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000233592, 2024-04-25T01:47:41.573 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000213 2024-04-25T01:47:41.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.573 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:47:41.573 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 181856 2024-04-25T01:47:41.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.574 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:47:41.574 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 7548, 2024-04-25T01:47:41.574 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 12373, 2024-04-25T01:47:41.574 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 3335420, 2024-04-25T01:47:41.574 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 5500232, 2024-04-25T01:47:41.574 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 17, 2024-04-25T01:47:41.574 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 9, 2024-04-25T01:47:41.574 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.596600853, 2024-04-25T01:47:41.574 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.241435707, 2024-04-25T01:47:41.574 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.372276648, 2024-04-25T01:47:41.575 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.070644483, 2024-04-25T01:47:41.575 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:41.575 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 12373, 2024-04-25T01:47:41.575 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.303556006, 2024-04-25T01:47:41.575 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000024533 2024-04-25T01:47:41.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.576 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:41.576 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 14407, 2024-04-25T01:47:41.576 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.002178860, 2024-04-25T01:47:41.576 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000151 2024-04-25T01:47:41.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.576 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 2746496, 2024-04-25T01:47:41.576 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 4463152 2024-04-25T01:47:41.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.576 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-04-25T01:47:41.576 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-04-25T01:47:41.576 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-04-25T01:47:41.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.576 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-04-25T01:47:41.577 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T01:47:41.577 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T01:47:41.577 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 4478, 2024-04-25T01:47:41.577 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 2.462811612, 2024-04-25T01:47:41.577 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000549980 2024-04-25T01:47:41.577 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:41.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.577 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-04-25T01:47:41.577 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T01:47:41.577 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T01:47:41.577 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 5481, 2024-04-25T01:47:41.577 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.265226783, 2024-04-25T01:47:41.577 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000048390 2024-04-25T01:47:41.577 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:41.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.578 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-04-25T01:47:41.578 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 2483, 2024-04-25T01:47:41.578 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 2483, 2024-04-25T01:47:41.578 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-04-25T01:47:41.578 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2483, 2024-04-25T01:47:41.578 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 3.222690899, 2024-04-25T01:47:41.578 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001297902 2024-04-25T01:47:41.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.578 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-04-25T01:47:41.578 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 0, 2024-04-25T01:47:41.578 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-04-25T01:47:41.578 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-04-25T01:47:41.578 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 12, 2024-04-25T01:47:41.579 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-04-25T01:47:41.579 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-04-25T01:47:41.579 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 24, 2024-04-25T01:47:41.579 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 0, 2024-04-25T01:47:41.579 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 24, 2024-04-25T01:47:41.579 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-04-25T01:47:41.579 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 23, 2024-04-25T01:47:41.579 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2024-04-25T01:47:41.579 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 1, 2024-04-25T01:47:41.579 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 1, 2024-04-25T01:47:41.579 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 2, 2024-04-25T01:47:41.579 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 2517, 2024-04-25T01:47:41.579 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 2503, 2024-04-25T01:47:41.580 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-04-25T01:47:41.580 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 0, 2024-04-25T01:47:41.580 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:47:41.580 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-04-25T01:47:41.580 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-04-25T01:47:41.580 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-04-25T01:47:41.580 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-04-25T01:47:41.580 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-04-25T01:47:41.580 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-04-25T01:47:41.580 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-04-25T01:47:41.580 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-04-25T01:47:41.580 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-04-25T01:47:41.580 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:47:41.580 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-04-25T01:47:41.581 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 4, 2024-04-25T01:47:41.581 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 234, 2024-04-25T01:47:41.581 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 0, 2024-04-25T01:47:41.581 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:47:41.581 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-04-25T01:47:41.581 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:47:41.581 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:47:41.581 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:47:41.581 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:47:41.581 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:47:41.581 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-04-25T01:47:41.581 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-04-25T01:47:41.582 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 20, 2024-04-25T01:47:41.582 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 3639, 2024-04-25T01:47:41.582 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:47:41.582 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:47:41.582 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:47:41.582 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 2437, 2024-04-25T01:47:41.582 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 1212, 2024-04-25T01:47:41.582 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 12, 2024-04-25T01:47:41.582 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 1242 2024-04-25T01:47:41.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.582 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-04-25T01:47:41.582 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 21, 2024-04-25T01:47:41.582 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 0, 2024-04-25T01:47:41.582 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 0, 2024-04-25T01:47:41.583 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 0, 2024-04-25T01:47:41.583 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 36, 2024-04-25T01:47:41.583 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-04-25T01:47:41.583 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-04-25T01:47:41.583 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:47:41.583 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 1212, 2024-04-25T01:47:41.583 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 1212, 2024-04-25T01:47:41.583 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-04-25T01:47:41.583 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-04-25T01:47:41.583 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-04-25T01:47:41.583 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:47:41.583 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:47:41.583 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-04-25T01:47:41.583 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-04-25T01:47:41.584 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:47:41.584 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:47:41.584 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:47:41.584 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-04-25T01:47:41.584 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-04-25T01:47:41.584 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:47:41.584 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-04-25T01:47:41.584 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-04-25T01:47:41.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.584 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-04-25T01:47:41.584 INFO:teuthology.orchestra.run.smithi152.stdout: "evlrg": 0, 2024-04-25T01:47:41.584 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 6115, 2024-04-25T01:47:41.584 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-04-25T01:47:41.584 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-04-25T01:47:41.585 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 6115, 2024-04-25T01:47:41.585 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-04-25T01:47:41.585 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-04-25T01:47:41.585 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 9, 2024-04-25T01:47:41.585 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-04-25T01:47:41.585 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-04-25T01:47:41.585 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 9, 2024-04-25T01:47:41.585 INFO:teuthology.orchestra.run.smithi152.stdout: "segmjr": 2, 2024-04-25T01:47:41.585 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-04-25T01:47:41.585 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-04-25T01:47:41.585 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 4194304, 2024-04-25T01:47:41.586 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 16838700, 2024-04-25T01:47:41.586 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 0, 2024-04-25T01:47:41.586 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-04-25T01:47:41.586 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2439, 2024-04-25T01:47:41.586 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 3.895569703, 2024-04-25T01:47:41.586 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001597199 2024-04-25T01:47:41.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.586 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 0 2024-04-25T01:47:41.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.586 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-04-25T01:47:41.586 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 23, 2024-04-25T01:47:41.586 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 1235, 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 1212, 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 18, 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 24, 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 6, 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 24, 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 1243, 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 1219, 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 1, 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 1226, 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 1225, 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 58980, 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 231804 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-04-25T01:47:41.587 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 2737, 2024-04-25T01:47:41.588 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 1, 2024-04-25T01:47:41.588 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 2483, 2024-04-25T01:47:41.588 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 17, 2024-04-25T01:47:41.588 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 2, 2024-04-25T01:47:41.588 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-04-25T01:47:41.588 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1210, 2024-04-25T01:47:41.588 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 1.293636433, 2024-04-25T01:47:41.588 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001069121 2024-04-25T01:47:41.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.588 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-04-25T01:47:41.588 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 28, 2024-04-25T01:47:41.588 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.007148900, 2024-04-25T01:47:41.588 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000255317 2024-04-25T01:47:41.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.589 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-04-25T01:47:41.589 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.589 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.589 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.589 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-04-25T01:47:41.589 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.589 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.589 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.589 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-04-25T01:47:41.589 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1, 2024-04-25T01:47:41.589 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000260693, 2024-04-25T01:47:41.590 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000260693 2024-04-25T01:47:41.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.590 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-04-25T01:47:41.590 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.590 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.590 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.590 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-04-25T01:47:41.590 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.591 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.591 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.591 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-04-25T01:47:41.591 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.591 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.591 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.591 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-04-25T01:47:41.591 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.591 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.591 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.592 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-04-25T01:47:41.592 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.592 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.592 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.592 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-04-25T01:47:41.592 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.592 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.593 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.593 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-04-25T01:47:41.593 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 12, 2024-04-25T01:47:41.593 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.020593704, 2024-04-25T01:47:41.593 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001716142 2024-04-25T01:47:41.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.594 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-04-25T01:47:41.594 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.594 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.594 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.594 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-04-25T01:47:41.594 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.594 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.594 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.594 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-04-25T01:47:41.594 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1, 2024-04-25T01:47:41.594 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000204453, 2024-04-25T01:47:41.595 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000204453 2024-04-25T01:47:41.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.595 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-04-25T01:47:41.595 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 6, 2024-04-25T01:47:41.595 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.010107163, 2024-04-25T01:47:41.595 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001684527 2024-04-25T01:47:41.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.595 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-04-25T01:47:41.595 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2, 2024-04-25T01:47:41.595 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.002999696, 2024-04-25T01:47:41.595 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001499848 2024-04-25T01:47:41.596 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.596 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-04-25T01:47:41.596 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.596 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.596 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.596 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.596 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-04-25T01:47:41.596 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.596 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.596 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.596 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.596 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-04-25T01:47:41.596 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 6, 2024-04-25T01:47:41.596 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.039236432, 2024-04-25T01:47:41.597 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.006539405 2024-04-25T01:47:41.597 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.597 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-04-25T01:47:41.597 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.597 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.597 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.597 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.597 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-04-25T01:47:41.597 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.597 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.597 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.597 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.597 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-04-25T01:47:41.598 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2, 2024-04-25T01:47:41.598 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.001390006, 2024-04-25T01:47:41.598 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000695003 2024-04-25T01:47:41.598 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.598 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-04-25T01:47:41.598 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.598 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.598 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.598 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.599 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-04-25T01:47:41.599 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.599 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.599 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.599 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.599 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-04-25T01:47:41.599 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.599 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.599 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.599 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.599 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-04-25T01:47:41.599 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 9, 2024-04-25T01:47:41.599 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.021385006, 2024-04-25T01:47:41.599 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.002376111 2024-04-25T01:47:41.600 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.600 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-04-25T01:47:41.600 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.600 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.600 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.600 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.600 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-04-25T01:47:41.600 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1206, 2024-04-25T01:47:41.600 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 1.825728413, 2024-04-25T01:47:41.600 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001513870 2024-04-25T01:47:41.600 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.600 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:47:41.600 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:47:41.601 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-04-25T01:47:41.601 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.601 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.601 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.601 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:41.601 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.601 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-04-25T01:47:41.601 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 1, 2024-04-25T01:47:41.601 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 4, 2024-04-25T01:47:41.601 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 3, 2024-04-25T01:47:41.601 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 1, 2024-04-25T01:47:41.601 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-04-25T01:47:41.601 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 2129, 2024-04-25T01:47:41.602 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 2129, 2024-04-25T01:47:41.602 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 119, 2024-04-25T01:47:41.602 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-04-25T01:47:41.602 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.602 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-04-25T01:47:41.602 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:47:41.602 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-04-25T01:47:41.602 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:47:41.602 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:47:41.602 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:47:41.602 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:47:41.603 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:47:41.603 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:47:41.603 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:47:41.603 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:47:41.603 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:47:41.603 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:47:41.603 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:47:41.603 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:47:41.603 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:47:41.603 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-04-25T01:47:41.603 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:47:41.603 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-04-25T01:47:41.604 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:47:41.604 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:47:41.604 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:47:41.604 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:47:41.604 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:47:41.604 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:47:41.604 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:47:41.604 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-04-25T01:47:41.604 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:47:41.604 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:47:41.605 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:47:41.605 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-04-25T01:47:41.605 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-04-25T01:47:41.605 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-04-25T01:47:41.605 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:47:41.605 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:47:41.605 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:47:41.605 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:47:41.605 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 324115, 2024-04-25T01:47:41.605 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 142, 2024-04-25T01:47:41.605 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 528, 2024-04-25T01:47:41.605 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 6, 2024-04-25T01:47:41.606 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-04-25T01:47:41.606 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-04-25T01:47:41.606 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:47:41.606 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-04-25T01:47:41.606 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:47:41.606 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-04-25T01:47:41.606 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 9096, 2024-04-25T01:47:41.606 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 176, 2024-04-25T01:47:41.606 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:47:41.606 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:47:41.606 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-04-25T01:47:41.606 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-04-25T01:47:41.607 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 97202, 2024-04-25T01:47:41.607 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 722, 2024-04-25T01:47:41.607 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-04-25T01:47:41.607 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-04-25T01:47:41.607 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-04-25T01:47:41.607 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-04-25T01:47:41.607 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.607 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-04-25T01:47:41.607 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-04-25T01:47:41.607 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-04-25T01:47:41.607 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 7823, 2024-04-25T01:47:41.607 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 12895295, 2024-04-25T01:47:41.607 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 32, 2024-04-25T01:47:41.608 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 7823, 2024-04-25T01:47:41.608 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-04-25T01:47:41.608 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 7823, 2024-04-25T01:47:41.608 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 35.565577863, 2024-04-25T01:47:41.608 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.004546283 2024-04-25T01:47:41.608 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.608 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-04-25T01:47:41.608 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-04-25T01:47:41.608 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 7823, 2024-04-25T01:47:41.609 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 7903 2024-04-25T01:47:41.609 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.609 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 7823, 2024-04-25T01:47:41.609 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 32, 2024-04-25T01:47:41.609 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 7791, 2024-04-25T01:47:41.609 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-04-25T01:47:41.609 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-04-25T01:47:41.609 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 0, 2024-04-25T01:47:41.609 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 11, 2024-04-25T01:47:41.610 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 32, 2024-04-25T01:47:41.610 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 2471, 2024-04-25T01:47:41.610 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 31, 2024-04-25T01:47:41.610 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-04-25T01:47:41.610 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-04-25T01:47:41.610 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-04-25T01:47:41.610 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-04-25T01:47:41.610 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 5219, 2024-04-25T01:47:41.610 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-04-25T01:47:41.610 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-04-25T01:47:41.610 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-04-25T01:47:41.610 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 0, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 22, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-04-25T01:47:41.611 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-04-25T01:47:41.612 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-04-25T01:47:41.612 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-04-25T01:47:41.612 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-04-25T01:47:41.612 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-04-25T01:47:41.612 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-04-25T01:47:41.612 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-04-25T01:47:41.612 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-04-25T01:47:41.612 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-04-25T01:47:41.612 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-04-25T01:47:41.612 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-04-25T01:47:41.612 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-04-25T01:47:41.612 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-04-25T01:47:41.612 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 94, 2024-04-25T01:47:41.612 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 9, 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 12, 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 12, 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 98, 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 0, 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 19 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 1, 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-04-25T01:47:41.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.614 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-04-25T01:47:41.614 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-04-25T01:47:41.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.614 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-04-25T01:47:41.614 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-04-25T01:47:41.614 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 12, 2024-04-25T01:47:41.614 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-04-25T01:47:41.614 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 4, 2024-04-25T01:47:41.614 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed_ops": 4704, 2024-04-25T01:47:41.614 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 1212, 2024-04-25T01:47:41.614 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-04-25T01:47:41.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.614 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 14887, 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 3720954, 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 14887, 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 14887, 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 3720954, 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.615 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.616 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:41.616 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.616 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-04-25T01:47:41.616 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:41.616 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T01:47:41.616 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:41.616 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T01:47:41.616 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T01:47:41.616 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:41.616 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:41.616 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 7823, 2024-04-25T01:47:41.616 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 13027355, 2024-04-25T01:47:41.616 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 7823, 2024-04-25T01:47:41.617 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 13027355, 2024-04-25T01:47:41.617 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:41.617 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.617 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.617 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.617 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:41.617 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.617 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-04-25T01:47:41.617 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:41.617 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-04-25T01:47:41.617 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:41.617 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T01:47:41.617 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T01:47:41.617 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:41.618 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:41.618 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 7823, 2024-04-25T01:47:41.618 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 7823, 2024-04-25T01:47:41.618 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 7823, 2024-04-25T01:47:41.618 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 7823, 2024-04-25T01:47:41.618 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:41.618 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.618 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.619 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.619 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:41.619 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.619 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:47:41.619 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:41.619 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T01:47:41.619 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:41.619 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 1212, 2024-04-25T01:47:41.619 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 132060, 2024-04-25T01:47:41.619 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:41.619 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 1212, 2024-04-25T01:47:41.619 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T01:47:41.620 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T01:47:41.620 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 32, 2024-04-25T01:47:41.620 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 132060, 2024-04-25T01:47:41.620 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:41.620 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.620 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.620 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.621 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:41.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:41.621 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x56441a7e2500": { 2024-04-25T01:47:41.621 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:41.621 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T01:47:41.621 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:41.621 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 6115, 2024-04-25T01:47:41.621 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 12644396, 2024-04-25T01:47:41.621 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:41.621 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 6115, 2024-04-25T01:47:41.621 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T01:47:41.621 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T01:47:41.621 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 2439, 2024-04-25T01:47:41.622 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 12644396, 2024-04-25T01:47:41.622 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:41.622 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:41.622 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:41.622 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:41.622 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:41.622 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:41.622 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-25T01:47:42.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:42 smithi152 ceph-mon[32465]: pgmap v426: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 KiB/s wr, 28 op/s 2024-04-25T01:47:42.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:42 smithi196 ceph-mon[32978]: pgmap v426: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 KiB/s wr, 28 op/s 2024-04-25T01:47:42.495 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T01:47:42.495 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T01:47:42.495 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=21 2024-04-25T01:47:42.495 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-25T01:47:42.495 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T01:47:42.495 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T01:47:42.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:42 smithi078 ceph-mon[29469]: pgmap v426: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 18 KiB/s wr, 28 op/s 2024-04-25T01:47:42.742 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:47:44.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:44 smithi196 ceph-mon[32978]: pgmap v427: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 15 op/s 2024-04-25T01:47:44.392 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:44 smithi152 ceph-mon[32465]: pgmap v427: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 15 op/s 2024-04-25T01:47:44.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:44 smithi078 ceph-mon[29469]: pgmap v427: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 KiB/s wr, 15 op/s 2024-04-25T01:47:44.520 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-25T01:47:44.520 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:47:44.520 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 668, 2024-04-25T01:47:44.520 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 670, 2024-04-25T01:47:44.520 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 192563, 2024-04-25T01:47:44.520 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 212431, 2024-04-25T01:47:44.520 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 6, 2024-04-25T01:47:44.520 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 2, 2024-04-25T01:47:44.520 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.052076673, 2024-04-25T01:47:44.520 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.016356082, 2024-04-25T01:47:44.520 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.012828345, 2024-04-25T01:47:44.521 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.018159980, 2024-04-25T01:47:44.521 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:44.521 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 668, 2024-04-25T01:47:44.521 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.021197816, 2024-04-25T01:47:44.521 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000031733 2024-04-25T01:47:44.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.521 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:44.521 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.521 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.521 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.521 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:47:44.521 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:47:44.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.522 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:47:44.522 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 7, 2024-04-25T01:47:44.522 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 66, 2024-04-25T01:47:44.522 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 3788, 2024-04-25T01:47:44.522 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 176570, 2024-04-25T01:47:44.522 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 4, 2024-04-25T01:47:44.522 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 3, 2024-04-25T01:47:44.522 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.004623292, 2024-04-25T01:47:44.522 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.002407310, 2024-04-25T01:47:44.522 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.001570717, 2024-04-25T01:47:44.522 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000050932, 2024-04-25T01:47:44.522 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:44.523 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 65, 2024-04-25T01:47:44.523 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.004882559, 2024-04-25T01:47:44.523 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000075116 2024-04-25T01:47:44.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.523 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:44.523 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.523 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.523 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.523 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-04-25T01:47:44.523 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 175872 2024-04-25T01:47:44.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.523 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:47:44.524 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 103, 2024-04-25T01:47:44.524 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 92, 2024-04-25T01:47:44.524 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 766476, 2024-04-25T01:47:44.524 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 51326, 2024-04-25T01:47:44.524 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 3, 2024-04-25T01:47:44.524 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 2, 2024-04-25T01:47:44.524 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.018577430, 2024-04-25T01:47:44.524 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.005526290, 2024-04-25T01:47:44.524 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.015608049, 2024-04-25T01:47:44.524 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.001902063, 2024-04-25T01:47:44.524 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:44.524 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 91, 2024-04-25T01:47:44.524 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.005235334, 2024-04-25T01:47:44.525 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000057531 2024-04-25T01:47:44.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.525 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:44.525 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.525 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.525 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.525 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 766208, 2024-04-25T01:47:44.525 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 51008 2024-04-25T01:47:44.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.525 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-04-25T01:47:44.525 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-04-25T01:47:44.525 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-04-25T01:47:44.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.526 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-04-25T01:47:44.526 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T01:47:44.526 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T01:47:44.526 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1153, 2024-04-25T01:47:44.526 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.028682947, 2024-04-25T01:47:44.526 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000024876 2024-04-25T01:47:44.526 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:44.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.526 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-04-25T01:47:44.526 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T01:47:44.527 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T01:47:44.527 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.527 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.527 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.527 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:44.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.527 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-04-25T01:47:44.527 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 0, 2024-04-25T01:47:44.527 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 0, 2024-04-25T01:47:44.527 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-04-25T01:47:44.527 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.528 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.528 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.528 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-04-25T01:47:44.528 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 0, 2024-04-25T01:47:44.528 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-04-25T01:47:44.528 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-04-25T01:47:44.528 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 0, 2024-04-25T01:47:44.528 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-04-25T01:47:44.528 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-04-25T01:47:44.528 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 0, 2024-04-25T01:47:44.528 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 0, 2024-04-25T01:47:44.528 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 0, 2024-04-25T01:47:44.529 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-04-25T01:47:44.529 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 0, 2024-04-25T01:47:44.529 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2024-04-25T01:47:44.529 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 0, 2024-04-25T01:47:44.529 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 0, 2024-04-25T01:47:44.529 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 1, 2024-04-25T01:47:44.529 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 0, 2024-04-25T01:47:44.529 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 0, 2024-04-25T01:47:44.529 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-04-25T01:47:44.529 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 0, 2024-04-25T01:47:44.529 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:47:44.529 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-04-25T01:47:44.529 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-04-25T01:47:44.529 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-04-25T01:47:44.530 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-04-25T01:47:44.530 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-04-25T01:47:44.530 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-04-25T01:47:44.530 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-04-25T01:47:44.530 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-04-25T01:47:44.530 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-04-25T01:47:44.530 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:47:44.530 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-04-25T01:47:44.530 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 0, 2024-04-25T01:47:44.530 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 0, 2024-04-25T01:47:44.530 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 0, 2024-04-25T01:47:44.530 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:47:44.530 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-04-25T01:47:44.531 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:47:44.531 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:47:44.531 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:47:44.531 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:47:44.531 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:47:44.531 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-04-25T01:47:44.531 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-04-25T01:47:44.531 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T01:47:44.531 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-04-25T01:47:44.531 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:47:44.531 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:47:44.531 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:47:44.531 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-04-25T01:47:44.531 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-04-25T01:47:44.532 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-04-25T01:47:44.532 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 0 2024-04-25T01:47:44.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.532 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-04-25T01:47:44.532 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-04-25T01:47:44.532 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 0, 2024-04-25T01:47:44.532 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 0, 2024-04-25T01:47:44.532 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 0, 2024-04-25T01:47:44.532 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-04-25T01:47:44.532 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-04-25T01:47:44.532 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-04-25T01:47:44.532 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:47:44.532 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-04-25T01:47:44.532 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-04-25T01:47:44.533 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-04-25T01:47:44.533 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-04-25T01:47:44.533 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-04-25T01:47:44.533 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:47:44.533 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:47:44.533 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-04-25T01:47:44.533 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-04-25T01:47:44.533 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:47:44.533 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:47:44.533 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:47:44.533 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-04-25T01:47:44.533 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-04-25T01:47:44.533 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:47:44.533 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-04-25T01:47:44.534 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-04-25T01:47:44.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.534 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-04-25T01:47:44.534 INFO:teuthology.orchestra.run.smithi152.stdout: "evlrg": 0, 2024-04-25T01:47:44.534 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 0, 2024-04-25T01:47:44.534 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-04-25T01:47:44.534 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-04-25T01:47:44.534 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 7, 2024-04-25T01:47:44.534 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-04-25T01:47:44.534 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-04-25T01:47:44.534 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 0, 2024-04-25T01:47:44.534 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-04-25T01:47:44.534 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-04-25T01:47:44.535 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 2, 2024-04-25T01:47:44.535 INFO:teuthology.orchestra.run.smithi152.stdout: "segmjr": 2, 2024-04-25T01:47:44.535 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-04-25T01:47:44.535 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-04-25T01:47:44.535 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 4194304, 2024-04-25T01:47:44.535 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 4196264, 2024-04-25T01:47:44.535 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 4196123, 2024-04-25T01:47:44.535 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-04-25T01:47:44.535 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.535 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.535 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.535 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 7 2024-04-25T01:47:44.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.536 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-04-25T01:47:44.536 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 3, 2024-04-25T01:47:44.536 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 3, 2024-04-25T01:47:44.536 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 0, 2024-04-25T01:47:44.536 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 1, 2024-04-25T01:47:44.536 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 1, 2024-04-25T01:47:44.536 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 0, 2024-04-25T01:47:44.536 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 0, 2024-04-25T01:47:44.536 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 0, 2024-04-25T01:47:44.536 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 0, 2024-04-25T01:47:44.536 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 0, 2024-04-25T01:47:44.536 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 0, 2024-04-25T01:47:44.536 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 0, 2024-04-25T01:47:44.537 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 49436, 2024-04-25T01:47:44.537 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 215420 2024-04-25T01:47:44.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.537 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-04-25T01:47:44.537 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 0, 2024-04-25T01:47:44.537 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-04-25T01:47:44.537 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 0, 2024-04-25T01:47:44.537 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 0, 2024-04-25T01:47:44.537 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2024-04-25T01:47:44.537 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-04-25T01:47:44.537 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.537 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.537 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.538 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-04-25T01:47:44.538 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.538 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.538 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.538 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-04-25T01:47:44.538 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.538 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.538 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.538 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-04-25T01:47:44.538 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.539 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.539 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.539 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-04-25T01:47:44.539 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.539 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.539 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.539 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-04-25T01:47:44.539 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.539 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.539 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.540 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-04-25T01:47:44.540 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.540 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.540 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.540 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-04-25T01:47:44.540 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.540 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.540 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.540 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-04-25T01:47:44.540 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.540 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.540 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.541 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-04-25T01:47:44.541 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.541 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.541 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.541 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-04-25T01:47:44.541 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.541 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.541 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.541 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-04-25T01:47:44.541 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.541 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.542 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.542 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-04-25T01:47:44.542 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.542 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.542 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.543 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-04-25T01:47:44.543 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.543 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.543 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.543 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-04-25T01:47:44.543 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.543 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.543 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.543 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-04-25T01:47:44.543 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.544 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.544 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.544 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-04-25T01:47:44.544 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.544 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.544 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.544 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-04-25T01:47:44.544 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.544 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.544 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.545 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-04-25T01:47:44.545 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.545 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.545 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.545 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-04-25T01:47:44.545 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.545 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.545 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.545 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-04-25T01:47:44.545 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.546 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.546 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.546 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-04-25T01:47:44.546 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.546 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.546 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.546 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-04-25T01:47:44.546 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.546 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.546 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.547 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-04-25T01:47:44.547 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.547 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.547 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.547 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-04-25T01:47:44.547 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.547 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.547 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.547 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-04-25T01:47:44.547 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.548 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.548 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.548 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-04-25T01:47:44.548 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.548 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.548 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.548 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-04-25T01:47:44.548 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.548 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.548 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.549 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-04-25T01:47:44.549 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.549 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.549 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.549 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:47:44.549 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:47:44.549 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-04-25T01:47:44.549 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.549 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.549 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.549 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:44.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.549 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-04-25T01:47:44.550 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 1, 2024-04-25T01:47:44.550 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 3, 2024-04-25T01:47:44.550 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 2, 2024-04-25T01:47:44.550 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 2, 2024-04-25T01:47:44.550 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-04-25T01:47:44.550 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 0, 2024-04-25T01:47:44.550 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 0, 2024-04-25T01:47:44.550 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 2466, 2024-04-25T01:47:44.550 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-04-25T01:47:44.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.550 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-04-25T01:47:44.550 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:47:44.551 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-04-25T01:47:44.551 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:47:44.551 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:47:44.551 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:47:44.551 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:47:44.551 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:47:44.551 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:47:44.551 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:47:44.551 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:47:44.551 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:47:44.551 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:47:44.551 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:47:44.551 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:47:44.552 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:47:44.552 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-04-25T01:47:44.552 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:47:44.552 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-04-25T01:47:44.552 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:47:44.552 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:47:44.552 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:47:44.552 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:47:44.552 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:47:44.552 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:47:44.552 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:47:44.552 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-04-25T01:47:44.552 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:47:44.553 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:47:44.553 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:47:44.553 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-04-25T01:47:44.553 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-04-25T01:47:44.553 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-04-25T01:47:44.553 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:47:44.553 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:47:44.553 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:47:44.553 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:47:44.553 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 181159, 2024-04-25T01:47:44.553 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 44, 2024-04-25T01:47:44.553 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:47:44.553 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-04-25T01:47:44.553 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-04-25T01:47:44.554 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-04-25T01:47:44.554 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:47:44.554 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-04-25T01:47:44.554 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:47:44.554 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-04-25T01:47:44.554 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 9096, 2024-04-25T01:47:44.554 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 176, 2024-04-25T01:47:44.554 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:47:44.554 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:47:44.554 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-04-25T01:47:44.554 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-04-25T01:47:44.554 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 8464, 2024-04-25T01:47:44.554 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 13, 2024-04-25T01:47:44.555 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-04-25T01:47:44.555 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-04-25T01:47:44.555 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-04-25T01:47:44.555 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-04-25T01:47:44.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.555 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-04-25T01:47:44.555 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-04-25T01:47:44.555 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-04-25T01:47:44.555 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 671, 2024-04-25T01:47:44.555 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 21, 2024-04-25T01:47:44.555 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 3, 2024-04-25T01:47:44.555 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 671, 2024-04-25T01:47:44.555 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-04-25T01:47:44.555 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 671, 2024-04-25T01:47:44.556 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.361995690, 2024-04-25T01:47:44.556 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000539486 2024-04-25T01:47:44.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.556 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-04-25T01:47:44.556 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-04-25T01:47:44.556 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 671, 2024-04-25T01:47:44.556 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 672 2024-04-25T01:47:44.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.556 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 671, 2024-04-25T01:47:44.556 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 671, 2024-04-25T01:47:44.556 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 0, 2024-04-25T01:47:44.556 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-04-25T01:47:44.556 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-04-25T01:47:44.557 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 419, 2024-04-25T01:47:44.557 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 0, 2024-04-25T01:47:44.557 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 250, 2024-04-25T01:47:44.557 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 0, 2024-04-25T01:47:44.557 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 0, 2024-04-25T01:47:44.557 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-04-25T01:47:44.557 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-04-25T01:47:44.557 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-04-25T01:47:44.557 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-04-25T01:47:44.557 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 0, 2024-04-25T01:47:44.557 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-04-25T01:47:44.557 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-04-25T01:47:44.557 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-04-25T01:47:44.558 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 1, 2024-04-25T01:47:44.558 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 0, 2024-04-25T01:47:44.558 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:47:44.558 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-04-25T01:47:44.558 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:47:44.558 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-04-25T01:47:44.558 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-04-25T01:47:44.558 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-04-25T01:47:44.558 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:47:44.558 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-04-25T01:47:44.558 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:47:44.558 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-04-25T01:47:44.558 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-04-25T01:47:44.559 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-04-25T01:47:44.559 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-04-25T01:47:44.559 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-04-25T01:47:44.559 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-04-25T01:47:44.559 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-04-25T01:47:44.559 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-04-25T01:47:44.559 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-04-25T01:47:44.559 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-04-25T01:47:44.559 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-04-25T01:47:44.559 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-04-25T01:47:44.559 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-04-25T01:47:44.559 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-04-25T01:47:44.559 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-04-25T01:47:44.559 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-04-25T01:47:44.560 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-04-25T01:47:44.560 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 94, 2024-04-25T01:47:44.560 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-04-25T01:47:44.560 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 4, 2024-04-25T01:47:44.560 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 4, 2024-04-25T01:47:44.560 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 4, 2024-04-25T01:47:44.560 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2024-04-25T01:47:44.560 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-04-25T01:47:44.560 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 0, 2024-04-25T01:47:44.560 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 2, 2024-04-25T01:47:44.560 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 0 2024-04-25T01:47:44.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.560 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-04-25T01:47:44.560 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 0, 2024-04-25T01:47:44.561 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:47:44.561 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-04-25T01:47:44.561 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-04-25T01:47:44.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.561 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-04-25T01:47:44.561 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-04-25T01:47:44.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.561 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-04-25T01:47:44.561 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-04-25T01:47:44.561 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T01:47:44.561 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-04-25T01:47:44.561 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-04-25T01:47:44.561 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed_ops": 0, 2024-04-25T01:47:44.561 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-04-25T01:47:44.562 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-04-25T01:47:44.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.562 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:47:44.562 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:44.562 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T01:47:44.562 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:44.562 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 774, 2024-04-25T01:47:44.562 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 884967, 2024-04-25T01:47:44.562 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:44.562 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 774, 2024-04-25T01:47:44.562 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T01:47:44.562 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T01:47:44.562 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 774, 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 884967, 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:44.563 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:44.564 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 671, 2024-04-25T01:47:44.564 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 1965, 2024-04-25T01:47:44.564 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 671, 2024-04-25T01:47:44.564 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 1965, 2024-04-25T01:47:44.564 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:44.564 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.564 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.564 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.564 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:44.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.564 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-04-25T01:47:44.564 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:44.564 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-04-25T01:47:44.564 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:44.565 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T01:47:44.565 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T01:47:44.565 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:44.565 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:44.565 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 671, 2024-04-25T01:47:44.565 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 671, 2024-04-25T01:47:44.565 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 671, 2024-04-25T01:47:44.565 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 671, 2024-04-25T01:47:44.565 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:44.565 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.565 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.565 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.565 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:44.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.566 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:47:44.566 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:44.566 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T01:47:44.566 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:44.566 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T01:47:44.566 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T01:47:44.566 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:44.566 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:44.566 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T01:47:44.566 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T01:47:44.566 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-04-25T01:47:44.566 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-04-25T01:47:44.566 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:44.566 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.567 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.567 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.567 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:44.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:44.567 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x56222185c6e0": { 2024-04-25T01:47:44.567 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:44.567 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T01:47:44.567 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:44.567 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T01:47:44.567 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T01:47:44.567 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:44.567 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:44.567 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T01:47:44.568 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T01:47:44.568 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-04-25T01:47:44.568 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-04-25T01:47:44.568 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:44.568 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:44.568 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:44.568 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:44.568 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:44.568 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:44.568 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-25T01:47:45.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:45 smithi152 ceph-mon[32465]: pgmap v428: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T01:47:45.388 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T01:47:45.388 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T01:47:45.388 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-25T01:47:45.388 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-25T01:47:45.388 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T01:47:45.388 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T01:47:45.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:45 smithi078 ceph-mon[29469]: pgmap v428: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T01:47:45.620 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:45 smithi196 ceph-mon[32978]: pgmap v428: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-25T01:47:45.631 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:47:47.608 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-25T01:47:47.608 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:47:47.608 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 41, 2024-04-25T01:47:47.608 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 43, 2024-04-25T01:47:47.608 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 11993, 2024-04-25T01:47:47.608 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 62267, 2024-04-25T01:47:47.608 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 14, 2024-04-25T01:47:47.609 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 7, 2024-04-25T01:47:47.609 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.007754696, 2024-04-25T01:47:47.609 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.000792376, 2024-04-25T01:47:47.609 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.004850009, 2024-04-25T01:47:47.609 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000525586, 2024-04-25T01:47:47.609 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:47.609 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 40, 2024-04-25T01:47:47.609 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.015614880, 2024-04-25T01:47:47.609 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000390372 2024-04-25T01:47:47.609 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.609 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:47.609 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 18, 2024-04-25T01:47:47.609 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000002732, 2024-04-25T01:47:47.610 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000151 2024-04-25T01:47:47.610 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.610 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 4928, 2024-04-25T01:47:47.610 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 1920 2024-04-25T01:47:47.610 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.610 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:47:47.610 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 698, 2024-04-25T01:47:47.610 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 509, 2024-04-25T01:47:47.610 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 218695, 2024-04-25T01:47:47.610 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 316647, 2024-04-25T01:47:47.610 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 7, 2024-04-25T01:47:47.610 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-04-25T01:47:47.610 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.055354811, 2024-04-25T01:47:47.611 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.025193395, 2024-04-25T01:47:47.611 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.035032088, 2024-04-25T01:47:47.611 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.001487500, 2024-04-25T01:47:47.611 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:47.611 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 509, 2024-04-25T01:47:47.611 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.034660025, 2024-04-25T01:47:47.611 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000068094 2024-04-25T01:47:47.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.611 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:47.611 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1107, 2024-04-25T01:47:47.611 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000196227, 2024-04-25T01:47:47.611 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000177 2024-04-25T01:47:47.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 176864 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 146, 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 132, 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 780631, 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 65806, 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 6, 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 5, 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.024101807, 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.005859919, 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.021547599, 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.002021139, 2024-04-25T01:47:47.612 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:47.613 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 131, 2024-04-25T01:47:47.613 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.021362366, 2024-04-25T01:47:47.613 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000163071 2024-04-25T01:47:47.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.613 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:47.613 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 5, 2024-04-25T01:47:47.613 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000914, 2024-04-25T01:47:47.613 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000182 2024-04-25T01:47:47.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.613 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 778112, 2024-04-25T01:47:47.613 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 56176 2024-04-25T01:47:47.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.613 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-04-25T01:47:47.613 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-04-25T01:47:47.614 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-04-25T01:47:47.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.614 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-04-25T01:47:47.614 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T01:47:47.614 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T01:47:47.614 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 44, 2024-04-25T01:47:47.614 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.001789080, 2024-04-25T01:47:47.614 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000040660 2024-04-25T01:47:47.614 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:47.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.614 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-04-25T01:47:47.614 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2, 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000007996, 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000003998 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 1, 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 1, 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1, 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000088169, 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000088169 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.615 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-04-25T01:47:47.616 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 0, 2024-04-25T01:47:47.616 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-04-25T01:47:47.616 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-04-25T01:47:47.616 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 11, 2024-04-25T01:47:47.616 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-04-25T01:47:47.616 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-04-25T01:47:47.616 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 17, 2024-04-25T01:47:47.616 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 1, 2024-04-25T01:47:47.616 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 16, 2024-04-25T01:47:47.616 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-04-25T01:47:47.616 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 13, 2024-04-25T01:47:47.616 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 8, 2024-04-25T01:47:47.616 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 1, 2024-04-25T01:47:47.616 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 1, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 3, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 9, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 6, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 3, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-04-25T01:47:47.617 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-04-25T01:47:47.618 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:47:47.618 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-04-25T01:47:47.618 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 4, 2024-04-25T01:47:47.618 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 234, 2024-04-25T01:47:47.618 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 0, 2024-04-25T01:47:47.618 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:47:47.618 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-04-25T01:47:47.618 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:47:47.618 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:47:47.618 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:47:47.618 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:47:47.618 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:47:47.618 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-04-25T01:47:47.619 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-04-25T01:47:47.619 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T01:47:47.619 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-04-25T01:47:47.619 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:47:47.619 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:47:47.619 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:47:47.619 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-04-25T01:47:47.619 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-04-25T01:47:47.619 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-04-25T01:47:47.619 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 0 2024-04-25T01:47:47.619 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.619 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-04-25T01:47:47.620 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-04-25T01:47:47.620 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 15, 2024-04-25T01:47:47.620 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 9, 2024-04-25T01:47:47.620 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 10, 2024-04-25T01:47:47.620 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-04-25T01:47:47.620 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-04-25T01:47:47.620 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-04-25T01:47:47.620 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:47:47.620 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-04-25T01:47:47.620 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-04-25T01:47:47.620 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-04-25T01:47:47.620 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-04-25T01:47:47.620 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-04-25T01:47:47.620 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:47:47.621 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:47:47.621 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-04-25T01:47:47.621 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-04-25T01:47:47.621 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:47:47.621 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:47:47.621 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:47:47.621 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-04-25T01:47:47.621 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-04-25T01:47:47.621 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:47:47.621 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-04-25T01:47:47.622 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-04-25T01:47:47.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.622 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-04-25T01:47:47.622 INFO:teuthology.orchestra.run.smithi152.stdout: "evlrg": 0, 2024-04-25T01:47:47.622 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 7, 2024-04-25T01:47:47.622 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-04-25T01:47:47.622 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-04-25T01:47:47.622 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 7, 2024-04-25T01:47:47.622 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-04-25T01:47:47.623 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-04-25T01:47:47.623 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 2, 2024-04-25T01:47:47.623 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-04-25T01:47:47.623 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-04-25T01:47:47.623 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 2, 2024-04-25T01:47:47.623 INFO:teuthology.orchestra.run.smithi152.stdout: "segmjr": 2, 2024-04-25T01:47:47.623 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-04-25T01:47:47.623 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-04-25T01:47:47.623 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 4194304, 2024-04-25T01:47:47.623 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 4196264, 2024-04-25T01:47:47.623 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 0, 2024-04-25T01:47:47.623 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-04-25T01:47:47.623 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 6, 2024-04-25T01:47:47.623 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.011527571, 2024-04-25T01:47:47.624 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001921261 2024-04-25T01:47:47.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.624 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 0 2024-04-25T01:47:47.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.624 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-04-25T01:47:47.624 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 21, 2024-04-25T01:47:47.624 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 27, 2024-04-25T01:47:47.624 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 6, 2024-04-25T01:47:47.624 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 20, 2024-04-25T01:47:47.624 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 25, 2024-04-25T01:47:47.624 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 5, 2024-04-25T01:47:47.624 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 17, 2024-04-25T01:47:47.624 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 25, 2024-04-25T01:47:47.624 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 8, 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 1, 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 1, 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 0, 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 51704, 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 223612 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 1, 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 1, 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 10, 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.625 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.626 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.626 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-04-25T01:47:47.626 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.626 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.626 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.626 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-04-25T01:47:47.626 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.626 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.626 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.626 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-04-25T01:47:47.626 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.627 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.627 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.627 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-04-25T01:47:47.627 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.627 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.627 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.627 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-04-25T01:47:47.627 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.627 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.627 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.627 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-04-25T01:47:47.628 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1, 2024-04-25T01:47:47.628 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000088169, 2024-04-25T01:47:47.628 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000088169 2024-04-25T01:47:47.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.628 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-04-25T01:47:47.628 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.628 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.628 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.628 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-04-25T01:47:47.628 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.628 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.628 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.629 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-04-25T01:47:47.629 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.629 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.629 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.629 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-04-25T01:47:47.629 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.629 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.629 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.629 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-04-25T01:47:47.629 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.630 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.630 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.630 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.631 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-04-25T01:47:47.632 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.632 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.632 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.632 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-04-25T01:47:47.632 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.632 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.632 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.632 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-04-25T01:47:47.632 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.632 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.632 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.633 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-04-25T01:47:47.633 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.633 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.633 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.633 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-04-25T01:47:47.633 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.634 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.634 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.634 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-04-25T01:47:47.634 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.634 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.634 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.634 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-04-25T01:47:47.634 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.634 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.634 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.634 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-04-25T01:47:47.635 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.635 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.635 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.635 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-04-25T01:47:47.635 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.635 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.635 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.635 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-04-25T01:47:47.635 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.635 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.635 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.637 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-04-25T01:47:47.637 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.637 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.637 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.637 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:47:47.637 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:47:47.637 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-04-25T01:47:47.637 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.637 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.637 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.637 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:47.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.637 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-04-25T01:47:47.638 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 1, 2024-04-25T01:47:47.638 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 3, 2024-04-25T01:47:47.638 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 2, 2024-04-25T01:47:47.638 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 1, 2024-04-25T01:47:47.638 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-04-25T01:47:47.638 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 1, 2024-04-25T01:47:47.638 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 0, 2024-04-25T01:47:47.638 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 126, 2024-04-25T01:47:47.638 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-04-25T01:47:47.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.638 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-04-25T01:47:47.638 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:47:47.639 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-04-25T01:47:47.639 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:47:47.639 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:47:47.639 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:47:47.639 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:47:47.639 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:47:47.639 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:47:47.639 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:47:47.639 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:47:47.639 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:47:47.639 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:47:47.639 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:47:47.639 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:47:47.639 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:47:47.640 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-04-25T01:47:47.640 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:47:47.640 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-04-25T01:47:47.640 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:47:47.640 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:47:47.640 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:47:47.640 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:47:47.640 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:47:47.640 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 246583, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 81, 2024-04-25T01:47:47.641 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:47:47.642 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-04-25T01:47:47.642 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-04-25T01:47:47.642 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-04-25T01:47:47.642 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:47:47.642 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-04-25T01:47:47.642 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:47:47.642 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-04-25T01:47:47.642 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 9096, 2024-04-25T01:47:47.642 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 176, 2024-04-25T01:47:47.642 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:47:47.642 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:47:47.642 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-04-25T01:47:47.642 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-04-25T01:47:47.643 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 85133, 2024-04-25T01:47:47.643 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 335, 2024-04-25T01:47:47.643 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-04-25T01:47:47.643 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-04-25T01:47:47.643 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-04-25T01:47:47.643 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-04-25T01:47:47.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.643 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-04-25T01:47:47.643 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-04-25T01:47:47.643 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-04-25T01:47:47.643 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 44, 2024-04-25T01:47:47.643 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 12563, 2024-04-25T01:47:47.643 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 33, 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 44, 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 44, 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.246997075, 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.005613569 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 44, 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 67 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 44, 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 0, 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 44, 2024-04-25T01:47:47.644 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 0, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 10, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 0, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 6, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 8, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 6, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 0, 2024-04-25T01:47:47.645 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 20, 2024-04-25T01:47:47.646 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:47:47.646 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-04-25T01:47:47.646 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:47:47.646 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-04-25T01:47:47.646 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-04-25T01:47:47.646 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-04-25T01:47:47.646 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:47:47.646 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-04-25T01:47:47.646 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:47:47.646 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-04-25T01:47:47.646 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-04-25T01:47:47.646 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-04-25T01:47:47.647 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-04-25T01:47:47.647 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-04-25T01:47:47.647 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-04-25T01:47:47.647 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-04-25T01:47:47.647 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-04-25T01:47:47.647 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-04-25T01:47:47.647 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-04-25T01:47:47.647 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-04-25T01:47:47.647 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-04-25T01:47:47.647 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-04-25T01:47:47.647 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-04-25T01:47:47.649 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-04-25T01:47:47.649 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-04-25T01:47:47.649 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-04-25T01:47:47.649 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 94, 2024-04-25T01:47:47.649 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-04-25T01:47:47.649 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 4, 2024-04-25T01:47:47.649 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 9, 2024-04-25T01:47:47.649 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 9, 2024-04-25T01:47:47.649 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2024-04-25T01:47:47.649 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-04-25T01:47:47.649 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 16, 2024-04-25T01:47:47.650 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 0, 2024-04-25T01:47:47.650 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 1 2024-04-25T01:47:47.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.650 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-04-25T01:47:47.650 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 1, 2024-04-25T01:47:47.650 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:47:47.650 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-04-25T01:47:47.650 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-04-25T01:47:47.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.650 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-04-25T01:47:47.650 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-04-25T01:47:47.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.650 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed_ops": 0, 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 881, 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 921886, 2024-04-25T01:47:47.651 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:47.652 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 881, 2024-04-25T01:47:47.652 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T01:47:47.652 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T01:47:47.652 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 881, 2024-04-25T01:47:47.652 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 921886, 2024-04-25T01:47:47.652 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:47.652 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.652 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.652 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.652 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:47.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.652 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-04-25T01:47:47.652 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:47.652 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T01:47:47.653 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:47.653 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T01:47:47.653 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T01:47:47.653 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:47.653 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:47.653 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 44, 2024-04-25T01:47:47.653 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 12563, 2024-04-25T01:47:47.653 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 44, 2024-04-25T01:47:47.653 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 12563, 2024-04-25T01:47:47.653 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:47.653 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.653 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.653 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.653 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:47.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.654 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-04-25T01:47:47.654 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:47.654 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-04-25T01:47:47.654 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:47.654 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T01:47:47.654 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T01:47:47.654 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:47.654 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:47.654 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 44, 2024-04-25T01:47:47.654 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 44, 2024-04-25T01:47:47.654 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 44, 2024-04-25T01:47:47.654 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 44, 2024-04-25T01:47:47.654 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T01:47:47.655 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-04-25T01:47:47.656 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-04-25T01:47:47.656 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:47.656 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.656 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.656 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.656 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:47.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:47.656 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x560f2021c500": { 2024-04-25T01:47:47.656 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:47.656 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T01:47:47.656 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:47.656 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 7, 2024-04-25T01:47:47.656 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 1960, 2024-04-25T01:47:47.656 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:47.657 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 7, 2024-04-25T01:47:47.657 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T01:47:47.657 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T01:47:47.657 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 6, 2024-04-25T01:47:47.657 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 1960, 2024-04-25T01:47:47.657 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:47.657 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:47.657 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:47.657 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:47.657 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:47.657 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:47.657 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-25T01:47:48.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:48 smithi152 ceph-mon[32465]: pgmap v429: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-04-25T01:47:48.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:48 smithi196 ceph-mon[32978]: pgmap v429: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-04-25T01:47:48.435 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T01:47:48.435 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T01:47:48.435 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-25T01:47:48.435 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=15 2024-04-25T01:47:48.435 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T01:47:48.435 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T01:47:48.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:48 smithi078 ceph-mon[29469]: pgmap v429: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-04-25T01:47:48.674 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T01:47:50.253 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:50 smithi152 ceph-mon[32465]: pgmap v430: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T01:47:50.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:50 smithi196 ceph-mon[32978]: pgmap v430: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T01:47:50.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:50 smithi078 ceph-mon[29469]: pgmap v430: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T01:47:50.546 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-25T01:47:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:47:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 157, 2024-04-25T01:47:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 149, 2024-04-25T01:47:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 775635, 2024-04-25T01:47:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 67982, 2024-04-25T01:47:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 7, 2024-04-25T01:47:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 3, 2024-04-25T01:47:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.024004875, 2024-04-25T01:47:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.006244401, 2024-04-25T01:47:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.019407995, 2024-04-25T01:47:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.002705995, 2024-04-25T01:47:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 145, 2024-04-25T01:47:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.014038469, 2024-04-25T01:47:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000096817 2024-04-25T01:47:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 770816, 2024-04-25T01:47:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 58816 2024-04-25T01:47:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:47:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 698, 2024-04-25T01:47:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 506, 2024-04-25T01:47:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 221426, 2024-04-25T01:47:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 295608, 2024-04-25T01:47:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 7, 2024-04-25T01:47:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-04-25T01:47:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.053540818, 2024-04-25T01:47:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.024408242, 2024-04-25T01:47:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.033873107, 2024-04-25T01:47:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.001556951, 2024-04-25T01:47:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 508, 2024-04-25T01:47:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.024615606, 2024-04-25T01:47:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000048455 2024-04-25T01:47:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1103, 2024-04-25T01:47:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000257400, 2024-04-25T01:47:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000233 2024-04-25T01:47:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:47:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 179184 2024-04-25T01:47:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:47:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 33, 2024-04-25T01:47:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 35, 2024-04-25T01:47:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 8428, 2024-04-25T01:47:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 64803, 2024-04-25T01:47:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 11, 2024-04-25T01:47:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 7, 2024-04-25T01:47:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.005045619, 2024-04-25T01:47:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.000743010, 2024-04-25T01:47:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.003850081, 2024-04-25T01:47:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000317005, 2024-04-25T01:47:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 33, 2024-04-25T01:47:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.007845614, 2024-04-25T01:47:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000237745 2024-04-25T01:47:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 32, 2024-04-25T01:47:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000004381, 2024-04-25T01:47:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000136 2024-04-25T01:47:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 4112, 2024-04-25T01:47:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 4912 2024-04-25T01:47:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-04-25T01:47:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-04-25T01:47:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-04-25T01:47:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-04-25T01:47:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T01:47:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T01:47:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 57, 2024-04-25T01:47:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.001416830, 2024-04-25T01:47:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000024856 2024-04-25T01:47:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-04-25T01:47:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T01:47:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T01:47:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2, 2024-04-25T01:47:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000004164, 2024-04-25T01:47:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000002082 2024-04-25T01:47:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-04-25T01:47:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 7, 2024-04-25T01:47:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 4, 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 4, 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.002224182, 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000556045 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 3, 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 11, 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 17, 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 1, 2024-04-25T01:47:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 16, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 13, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 8, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 0, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 0, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 3, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 18, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 13, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 3, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-04-25T01:47:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 4, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 234, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 0, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:47:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:47:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-04-25T01:47:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-04-25T01:47:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T01:47:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-04-25T01:47:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:47:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:47:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:47:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-04-25T01:47:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-04-25T01:47:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-04-25T01:47:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 2 2024-04-25T01:47:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-04-25T01:47:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 13, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 8, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 9, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:47:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "evlrg": 0, 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 6, 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-04-25T01:47:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 6, 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 2, 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 2, 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "segmjr": 2, 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 4194304, 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 4195956, 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 0, 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 5, 2024-04-25T01:47:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.009533033, 2024-04-25T01:47:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001906606 2024-04-25T01:47:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 0 2024-04-25T01:47:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-04-25T01:47:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 21, 2024-04-25T01:47:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 27, 2024-04-25T01:47:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 6, 2024-04-25T01:47:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 20, 2024-04-25T01:47:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 24, 2024-04-25T01:47:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 4, 2024-04-25T01:47:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 17, 2024-04-25T01:47:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 25, 2024-04-25T01:47:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 8, 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 0, 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 4, 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 4, 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 52480, 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 223612 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 10, 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 7, 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 8, 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 1, 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2, 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.001629447, 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000814723 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-04-25T01:47:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2, 2024-04-25T01:47:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000594735, 2024-04-25T01:47:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000297367 2024-04-25T01:47:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-04-25T01:47:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-04-25T01:47:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-04-25T01:47:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-04-25T01:47:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-04-25T01:47:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-04-25T01:47:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-04-25T01:47:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.568 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.568 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.568 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-04-25T01:47:50.568 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.568 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.568 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-04-25T01:47:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-04-25T01:47:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-04-25T01:47:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-04-25T01:47:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-04-25T01:47:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-04-25T01:47:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-04-25T01:47:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-04-25T01:47:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-04-25T01:47:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-04-25T01:47:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-04-25T01:47:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-04-25T01:47:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-04-25T01:47:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-04-25T01:47:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-04-25T01:47:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-04-25T01:47:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:47:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:47:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-04-25T01:47:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-04-25T01:47:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 0, 2024-04-25T01:47:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 2, 2024-04-25T01:47:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 2, 2024-04-25T01:47:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 0, 2024-04-25T01:47:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-04-25T01:47:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 3, 2024-04-25T01:47:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 3, 2024-04-25T01:47:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 63, 2024-04-25T01:47:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-04-25T01:47:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-04-25T01:47:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:47:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-04-25T01:47:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:47:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:47:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:47:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:47:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:47:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:47:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:47:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:47:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:47:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:47:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:47:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:47:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:47:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-04-25T01:47:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:47:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-04-25T01:47:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:47:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:47:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:47:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:47:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:47:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:47:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:47:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-04-25T01:47:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:47:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:47:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:47:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-04-25T01:47:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-04-25T01:47:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-04-25T01:47:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:47:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:47:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:47:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:47:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 231687, 2024-04-25T01:47:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 81, 2024-04-25T01:47:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 88, 2024-04-25T01:47:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 1, 2024-04-25T01:47:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-04-25T01:47:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-04-25T01:47:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:47:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-04-25T01:47:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 9096, 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 176, 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 84845, 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 343, 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-04-25T01:47:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-04-25T01:47:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-04-25T01:47:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 44, 2024-04-25T01:47:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 12269, 2024-04-25T01:47:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 33, 2024-04-25T01:47:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 44, 2024-04-25T01:47:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-04-25T01:47:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 44, 2024-04-25T01:47:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.198997633, 2024-04-25T01:47:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.004522673 2024-04-25T01:47:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-04-25T01:47:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 44, 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 68 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 44, 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 0, 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 44, 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 0, 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 10, 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 0, 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 5, 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 8, 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-04-25T01:47:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-04-25T01:47:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-04-25T01:47:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-04-25T01:47:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 6, 2024-04-25T01:47:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-04-25T01:47:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-04-25T01:47:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-04-25T01:47:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 0, 2024-04-25T01:47:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 20, 2024-04-25T01:47:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:47:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-04-25T01:47:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:47:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-04-25T01:47:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-04-25T01:47:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-04-25T01:47:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-04-25T01:47:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-04-25T01:47:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-04-25T01:47:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-04-25T01:47:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-04-25T01:47:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-04-25T01:47:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 94, 2024-04-25T01:47:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-04-25T01:47:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 4, 2024-04-25T01:47:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 8, 2024-04-25T01:47:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 8, 2024-04-25T01:47:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2024-04-25T01:47:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-04-25T01:47:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 17, 2024-04-25T01:47:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 0, 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 2 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 1, 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T01:47:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed_ops": 0, 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 884, 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 915525, 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 884, 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T01:47:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T01:47:50.589 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 884, 2024-04-25T01:47:50.589 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 915525, 2024-04-25T01:47:50.589 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:50.589 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.589 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.589 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.589 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:50.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.589 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-04-25T01:47:50.589 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:50.589 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T01:47:50.589 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:50.589 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T01:47:50.589 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T01:47:50.590 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:50.590 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:50.590 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 44, 2024-04-25T01:47:50.590 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 12269, 2024-04-25T01:47:50.590 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 44, 2024-04-25T01:47:50.590 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 12269, 2024-04-25T01:47:50.590 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:50.590 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.590 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.590 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.590 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:50.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.590 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-04-25T01:47:50.590 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 44, 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 44, 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 44, 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 44, 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.591 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:50.592 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x5624efeb2500": { 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 6, 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 1652, 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 6, 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T01:47:50.593 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 5, 2024-04-25T01:47:50.594 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 1652, 2024-04-25T01:47:50.594 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T01:47:50.594 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T01:47:50.594 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T01:47:50.594 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T01:47:50.594 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:50.594 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T01:47:50.594 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-25T01:47:51.391 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-25T01:47:51.392 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-25T01:47:51.392 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-25T01:47:51.392 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=13 2024-04-25T01:47:51.392 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T01:47:51.392 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T01:47:51.641 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:47:52.304 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:52 smithi152 ceph-mon[32465]: pgmap v431: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T01:47:52.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:52 smithi196 ceph-mon[32978]: pgmap v431: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T01:47:52.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:52 smithi078 ceph-mon[29469]: pgmap v431: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-25T01:47:53.514 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-25T01:47:53.514 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:47:53.514 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 502, 2024-04-25T01:47:53.514 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 503, 2024-04-25T01:47:53.514 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 4344489, 2024-04-25T01:47:53.515 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 159328, 2024-04-25T01:47:53.515 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-04-25T01:47:53.515 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-04-25T01:47:53.515 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.052436546, 2024-04-25T01:47:53.515 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.016024135, 2024-04-25T01:47:53.515 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.015580035, 2024-04-25T01:47:53.515 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.015485219, 2024-04-25T01:47:53.515 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:53.515 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 501, 2024-04-25T01:47:53.515 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.021308206, 2024-04-25T01:47:53.515 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000042531 2024-04-25T01:47:53.515 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.515 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:53.515 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.516 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.516 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.516 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.516 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:47:53.516 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:47:53.516 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.516 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:47:53.516 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 93, 2024-04-25T01:47:53.516 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 153, 2024-04-25T01:47:53.516 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 4284475, 2024-04-25T01:47:53.516 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 208946, 2024-04-25T01:47:53.516 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 5, 2024-04-25T01:47:53.517 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-04-25T01:47:53.517 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.012330253, 2024-04-25T01:47:53.517 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.004711315, 2024-04-25T01:47:53.517 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.004654960, 2024-04-25T01:47:53.517 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.001850073, 2024-04-25T01:47:53.517 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:53.517 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 152, 2024-04-25T01:47:53.517 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.010564993, 2024-04-25T01:47:53.517 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000069506 2024-04-25T01:47:53.517 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.517 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:53.517 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.517 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.518 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.518 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-04-25T01:47:53.518 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 180896 2024-04-25T01:47:53.518 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.518 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:47:53.518 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 324, 2024-04-25T01:47:53.518 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 313, 2024-04-25T01:47:53.518 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 5015311, 2024-04-25T01:47:53.518 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 121289, 2024-04-25T01:47:53.518 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 3, 2024-04-25T01:47:53.519 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-04-25T01:47:53.519 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.039465531, 2024-04-25T01:47:53.519 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.012875419, 2024-04-25T01:47:53.519 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.019945384, 2024-04-25T01:47:53.519 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.008643556, 2024-04-25T01:47:53.519 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:53.519 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 312, 2024-04-25T01:47:53.519 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.012835457, 2024-04-25T01:47:53.519 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000041139 2024-04-25T01:47:53.519 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.519 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:53.519 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.520 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.520 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 766704, 2024-04-25T01:47:53.520 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 51328 2024-04-25T01:47:53.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.520 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-04-25T01:47:53.520 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-04-25T01:47:53.520 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-04-25T01:47:53.520 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.520 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-04-25T01:47:53.520 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-25T01:47:53.520 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-25T01:47:53.520 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1333, 2024-04-25T01:47:53.521 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.036159269, 2024-04-25T01:47:53.521 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000027126 2024-04-25T01:47:53.521 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:53.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.521 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-04-25T01:47:53.521 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-25T01:47:53.521 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-25T01:47:53.521 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.521 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.521 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.521 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:53.521 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.521 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-04-25T01:47:53.521 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-04-25T01:47:53.522 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-04-25T01:47:53.522 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-04-25T01:47:53.522 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.522 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.522 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.522 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.522 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-04-25T01:47:53.522 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-04-25T01:47:53.522 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-04-25T01:47:53.522 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-04-25T01:47:53.522 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 1232, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 1232, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 1232, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 2, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-04-25T01:47:53.523 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-04-25T01:47:53.524 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:47:53.524 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-04-25T01:47:53.524 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-04-25T01:47:53.524 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-04-25T01:47:53.524 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-04-25T01:47:53.524 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-04-25T01:47:53.524 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-04-25T01:47:53.524 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-04-25T01:47:53.524 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-04-25T01:47:53.524 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-04-25T01:47:53.524 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:47:53.524 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-04-25T01:47:53.524 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 4, 2024-04-25T01:47:53.524 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-04-25T01:47:53.525 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-04-25T01:47:53.525 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:47:53.525 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-04-25T01:47:53.525 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:47:53.525 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:47:53.525 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:47:53.525 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:47:53.525 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:47:53.525 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-04-25T01:47:53.525 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-04-25T01:47:53.525 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T01:47:53.525 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-04-25T01:47:53.525 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:47:53.526 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:47:53.526 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:47:53.526 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-04-25T01:47:53.526 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-04-25T01:47:53.526 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-04-25T01:47:53.526 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-04-25T01:47:53.526 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.526 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-04-25T01:47:53.526 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-04-25T01:47:53.526 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-04-25T01:47:53.526 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-04-25T01:47:53.526 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-04-25T01:47:53.526 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-04-25T01:47:53.527 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-04-25T01:47:53.527 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-04-25T01:47:53.527 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:47:53.527 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-04-25T01:47:53.527 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-04-25T01:47:53.527 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-04-25T01:47:53.527 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-04-25T01:47:53.527 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-04-25T01:47:53.527 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:47:53.527 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:47:53.527 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-04-25T01:47:53.527 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-04-25T01:47:53.527 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 6116, 2024-04-25T01:47:53.528 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-04-25T01:47:53.529 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-04-25T01:47:53.529 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-04-25T01:47:53.529 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-04-25T01:47:53.529 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-04-25T01:47:53.529 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 9, 2024-04-25T01:47:53.529 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-04-25T01:47:53.529 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-04-25T01:47:53.529 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-04-25T01:47:53.529 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-04-25T01:47:53.529 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 16840352, 2024-04-25T01:47:53.529 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 16838700, 2024-04-25T01:47:53.529 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-04-25T01:47:53.529 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.529 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.530 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.530 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 6116 2024-04-25T01:47:53.530 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.530 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-04-25T01:47:53.530 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 20, 2024-04-25T01:47:53.530 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 1232, 2024-04-25T01:47:53.530 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 1212, 2024-04-25T01:47:53.530 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 15, 2024-04-25T01:47:53.530 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 21, 2024-04-25T01:47:53.530 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 6, 2024-04-25T01:47:53.530 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 1232, 2024-04-25T01:47:53.530 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 1239, 2024-04-25T01:47:53.531 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 7, 2024-04-25T01:47:53.531 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-04-25T01:47:53.531 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-04-25T01:47:53.531 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-04-25T01:47:53.531 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 54424, 2024-04-25T01:47:53.531 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215420 2024-04-25T01:47:53.531 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.531 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-04-25T01:47:53.531 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-04-25T01:47:53.531 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-04-25T01:47:53.531 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-04-25T01:47:53.531 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-04-25T01:47:53.531 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-04-25T01:47:53.531 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-04-25T01:47:53.532 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.532 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.532 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.532 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-04-25T01:47:53.532 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.532 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.532 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.532 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-04-25T01:47:53.532 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.532 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.532 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.532 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.533 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-04-25T01:47:53.533 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.533 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.533 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.533 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-04-25T01:47:53.533 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.533 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.533 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.533 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.534 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-04-25T01:47:53.534 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.534 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.534 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.534 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-04-25T01:47:53.534 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.534 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.534 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.534 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.534 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-04-25T01:47:53.534 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.535 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.535 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.535 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-04-25T01:47:53.535 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.535 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.535 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.535 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-04-25T01:47:53.535 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.535 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.535 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.535 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.535 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-04-25T01:47:53.536 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.536 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.536 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.536 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.536 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-04-25T01:47:53.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.537 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.537 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-04-25T01:47:53.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.537 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.537 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.537 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.537 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-04-25T01:47:53.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.538 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.538 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-04-25T01:47:53.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.538 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.538 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.538 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-04-25T01:47:53.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.538 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.538 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.539 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-04-25T01:47:53.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.539 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.539 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-04-25T01:47:53.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.539 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.539 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.539 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-04-25T01:47:53.539 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.540 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.540 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.540 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-04-25T01:47:53.540 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.540 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.540 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.540 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-04-25T01:47:53.540 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.540 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.540 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.540 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.541 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-04-25T01:47:53.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.541 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.541 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-04-25T01:47:53.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.541 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.541 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.541 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-04-25T01:47:53.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.541 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.541 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.542 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-04-25T01:47:53.542 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.542 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.542 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.542 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-04-25T01:47:53.542 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.542 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.542 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.542 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.542 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-04-25T01:47:53.543 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.543 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.543 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.543 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-04-25T01:47:53.543 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.543 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.543 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.543 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.543 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-04-25T01:47:53.543 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.543 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.543 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.544 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:47:53.544 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:47:53.544 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-04-25T01:47:53.544 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.544 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.544 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.544 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:53.544 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.544 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-04-25T01:47:53.544 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-04-25T01:47:53.544 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 4, 2024-04-25T01:47:53.544 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 3, 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 2, 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 2502, 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:47:53.545 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:47:53.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:47:53.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:47:53.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:47:53.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:47:53.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:47:53.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:47:53.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:47:53.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:47:53.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-04-25T01:47:53.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:47:53.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-04-25T01:47:53.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:47:53.546 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:47:53.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:47:53.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:47:53.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:47:53.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:47:53.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:47:53.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-04-25T01:47:53.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:47:53.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:47:53.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:47:53.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-04-25T01:47:53.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-04-25T01:47:53.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-04-25T01:47:53.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:47:53.547 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:47:53.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:47:53.548 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:47:53.548 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 189256, 2024-04-25T01:47:53.548 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 49, 2024-04-25T01:47:53.548 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:47:53.548 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-04-25T01:47:53.548 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-04-25T01:47:53.548 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-04-25T01:47:53.548 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:47:53.548 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-04-25T01:47:53.548 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:47:53.548 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-04-25T01:47:53.548 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9096, 2024-04-25T01:47:53.548 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 176, 2024-04-25T01:47:53.549 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:47:53.549 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:47:53.549 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-04-25T01:47:53.549 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-04-25T01:47:53.549 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 931050, 2024-04-25T01:47:53.549 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 5188, 2024-04-25T01:47:53.549 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-04-25T01:47:53.549 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-04-25T01:47:53.549 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-04-25T01:47:53.549 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-04-25T01:47:53.549 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.549 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-04-25T01:47:53.549 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-04-25T01:47:53.549 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-04-25T01:47:53.550 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 809, 2024-04-25T01:47:53.550 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 26, 2024-04-25T01:47:53.550 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 4, 2024-04-25T01:47:53.550 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 809, 2024-04-25T01:47:53.550 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-04-25T01:47:53.550 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 809, 2024-04-25T01:47:53.550 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 1.287971013, 2024-04-25T01:47:53.550 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001592053 2024-04-25T01:47:53.550 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.550 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-04-25T01:47:53.550 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-04-25T01:47:53.550 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 809, 2024-04-25T01:47:53.550 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 810 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 809, 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 809, 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 498, 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 308, 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-04-25T01:47:53.551 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 2, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-04-25T01:47:53.552 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:47:53.553 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-04-25T01:47:53.553 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-04-25T01:47:53.553 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-04-25T01:47:53.553 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-04-25T01:47:53.553 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-04-25T01:47:53.553 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-04-25T01:47:53.553 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-04-25T01:47:53.553 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-04-25T01:47:53.553 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-04-25T01:47:53.553 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-04-25T01:47:53.553 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-04-25T01:47:53.553 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-04-25T01:47:53.553 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-04-25T01:47:53.553 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-04-25T01:47:53.554 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-04-25T01:47:53.554 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-04-25T01:47:53.554 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-04-25T01:47:53.554 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 94, 2024-04-25T01:47:53.554 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-04-25T01:47:53.554 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 4, 2024-04-25T01:47:53.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 5, 2024-04-25T01:47:53.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 5, 2024-04-25T01:47:53.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-04-25T01:47:53.554 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-04-25T01:47:53.554 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-04-25T01:47:53.554 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-04-25T01:47:53.554 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-04-25T01:47:53.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.555 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-04-25T01:47:53.555 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-04-25T01:47:53.555 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:47:53.555 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-04-25T01:47:53.555 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-04-25T01:47:53.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.555 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-04-25T01:47:53.555 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-04-25T01:47:53.555 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.555 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-04-25T01:47:53.555 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-04-25T01:47:53.555 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T01:47:53.555 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 915, 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 13553627, 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 915, 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-25T01:47:53.556 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-25T01:47:53.557 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 915, 2024-04-25T01:47:53.557 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 13553627, 2024-04-25T01:47:53.557 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:53.557 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.557 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.557 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.557 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:53.557 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.557 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-04-25T01:47:53.557 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:53.557 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-25T01:47:53.557 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:53.557 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-25T01:47:53.557 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-25T01:47:53.558 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:53.558 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:53.558 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 809, 2024-04-25T01:47:53.558 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 12646709, 2024-04-25T01:47:53.558 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 809, 2024-04-25T01:47:53.558 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 12646709, 2024-04-25T01:47:53.558 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:53.558 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.558 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.558 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.558 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:53.558 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.558 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-04-25T01:47:53.559 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:53.559 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-04-25T01:47:53.559 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:53.559 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-25T01:47:53.559 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-25T01:47:53.559 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:53.559 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:53.559 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 809, 2024-04-25T01:47:53.559 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 809, 2024-04-25T01:47:53.559 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 809, 2024-04-25T01:47:53.559 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 809, 2024-04-25T01:47:53.559 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:53.559 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.559 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.560 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.560 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:53.560 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.560 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:47:53.560 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:53.560 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-25T01:47:53.560 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:53.560 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-25T01:47:53.560 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-25T01:47:53.560 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:53.560 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:53.560 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-25T01:47:53.560 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-25T01:47:53.561 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-25T01:47:53.561 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-25T01:47:53.561 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:53.561 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.561 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.561 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.561 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:53.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:53.561 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x5601eed5a6e0": { 2024-04-25T01:47:53.561 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:53.561 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-25T01:47:53.561 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:53.561 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-25T01:47:53.561 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-25T01:47:53.562 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:53.562 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:53.562 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-25T01:47:53.562 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-25T01:47:53.562 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-25T01:47:53.562 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-25T01:47:53.562 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:53.562 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:53.562 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:53.562 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:53.562 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:53.562 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:53.562 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-25T01:47:54.053 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:54 smithi152 ceph-mon[32465]: pgmap v432: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-25T01:47:54.053 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:54 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:47:54.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:54 smithi196 ceph-mon[32978]: pgmap v432: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-25T01:47:54.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:54 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:47:54.420 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T01:47:54.420 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T01:47:54.420 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T01:47:54.420 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-25T01:47:54.421 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T01:47:54.421 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T01:47:54.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:54 smithi078 ceph-mon[29469]: pgmap v432: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-25T01:47:54.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:54 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:47:54.668 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:47:55.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:55 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:47:55.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:55 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:47:55.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:55 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:47:55.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:55 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:47:55.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:55 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:47:55.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:55 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:47:55.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:47:55.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:47:55.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:55 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' 2024-04-25T01:47:56.302 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:56 smithi196 ceph-mon[32978]: pgmap v433: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T01:47:56.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:56 smithi152 ceph-mon[32465]: pgmap v433: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T01:47:56.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:56 smithi078 ceph-mon[29469]: pgmap v433: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T01:47:56.708 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-25T01:47:56.709 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:47:56.709 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 261, 2024-04-25T01:47:56.709 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 263, 2024-04-25T01:47:56.709 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 70385, 2024-04-25T01:47:56.709 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 83062, 2024-04-25T01:47:56.709 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-04-25T01:47:56.709 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-04-25T01:47:56.709 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.024505865, 2024-04-25T01:47:56.709 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.007333840, 2024-04-25T01:47:56.709 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.007001226, 2024-04-25T01:47:56.709 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.007221162, 2024-04-25T01:47:56.709 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:56.709 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 261, 2024-04-25T01:47:56.710 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.010678573, 2024-04-25T01:47:56.710 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000040914 2024-04-25T01:47:56.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.710 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:56.710 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.710 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.710 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.710 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:47:56.710 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:47:56.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.710 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:47:56.710 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 296, 2024-04-25T01:47:56.711 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 285, 2024-04-25T01:47:56.711 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 814096, 2024-04-25T01:47:56.711 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 113191, 2024-04-25T01:47:56.711 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 4, 2024-04-25T01:47:56.711 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-04-25T01:47:56.711 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.035819068, 2024-04-25T01:47:56.711 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.010368905, 2024-04-25T01:47:56.711 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.018399855, 2024-04-25T01:47:56.711 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.009086619, 2024-04-25T01:47:56.711 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:56.711 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 284, 2024-04-25T01:47:56.711 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.013325573, 2024-04-25T01:47:56.711 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000046921 2024-04-25T01:47:56.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.712 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:56.712 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.712 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.712 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.712 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 767088, 2024-04-25T01:47:56.712 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 52448 2024-04-25T01:47:56.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.712 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:47:56.712 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 259, 2024-04-25T01:47:56.712 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 320, 2024-04-25T01:47:56.712 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 88576, 2024-04-25T01:47:56.712 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 261844, 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 4, 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.033401852, 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.011240747, 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.008925056, 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.009880263, 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 319, 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.018783081, 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000058881 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.713 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.714 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-04-25T01:47:56.714 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 181072 2024-04-25T01:47:56.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.714 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-04-25T01:47:56.714 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-04-25T01:47:56.714 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-04-25T01:47:56.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.714 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-04-25T01:47:56.714 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-25T01:47:56.714 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-25T01:47:56.714 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1199, 2024-04-25T01:47:56.714 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.036725460, 2024-04-25T01:47:56.715 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000030630 2024-04-25T01:47:56.715 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:56.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.715 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-04-25T01:47:56.715 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-25T01:47:56.715 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-25T01:47:56.715 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.715 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.715 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.715 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:56.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.715 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-04-25T01:47:56.715 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-04-25T01:47:56.715 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-04-25T01:47:56.716 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-04-25T01:47:56.716 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.716 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.716 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.716 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.716 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-04-25T01:47:56.716 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-04-25T01:47:56.716 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-04-25T01:47:56.716 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-04-25T01:47:56.716 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-04-25T01:47:56.716 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-04-25T01:47:56.716 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-04-25T01:47:56.716 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 0, 2024-04-25T01:47:56.717 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-04-25T01:47:56.717 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 0, 2024-04-25T01:47:56.717 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-04-25T01:47:56.717 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 0, 2024-04-25T01:47:56.717 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-04-25T01:47:56.717 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-04-25T01:47:56.717 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-04-25T01:47:56.717 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 1, 2024-04-25T01:47:56.717 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-04-25T01:47:56.717 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-04-25T01:47:56.717 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-04-25T01:47:56.717 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-04-25T01:47:56.717 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:47:56.718 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-04-25T01:47:56.718 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-04-25T01:47:56.718 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-04-25T01:47:56.718 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-04-25T01:47:56.718 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-04-25T01:47:56.718 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-04-25T01:47:56.718 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-04-25T01:47:56.718 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-04-25T01:47:56.718 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-04-25T01:47:56.718 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:47:56.718 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-04-25T01:47:56.718 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 0, 2024-04-25T01:47:56.719 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 0, 2024-04-25T01:47:56.719 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-04-25T01:47:56.719 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:47:56.719 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-04-25T01:47:56.719 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:47:56.719 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:47:56.719 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:47:56.719 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:47:56.719 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:47:56.719 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-04-25T01:47:56.719 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-04-25T01:47:56.719 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T01:47:56.719 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-04-25T01:47:56.719 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:47:56.720 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:47:56.720 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:47:56.720 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-04-25T01:47:56.720 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-04-25T01:47:56.720 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-04-25T01:47:56.720 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-04-25T01:47:56.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.720 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-04-25T01:47:56.720 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-04-25T01:47:56.720 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-04-25T01:47:56.720 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-04-25T01:47:56.720 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-04-25T01:47:56.720 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-04-25T01:47:56.720 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-04-25T01:47:56.721 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-04-25T01:47:56.721 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:47:56.721 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-04-25T01:47:56.721 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-04-25T01:47:56.721 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-04-25T01:47:56.721 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-04-25T01:47:56.721 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-04-25T01:47:56.721 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:47:56.721 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:47:56.721 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-04-25T01:47:56.721 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-04-25T01:47:56.721 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:47:56.721 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:47:56.722 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:47:56.722 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-04-25T01:47:56.722 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-04-25T01:47:56.722 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:47:56.722 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-04-25T01:47:56.722 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-04-25T01:47:56.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.722 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-04-25T01:47:56.722 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-04-25T01:47:56.722 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-04-25T01:47:56.722 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-04-25T01:47:56.722 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-04-25T01:47:56.723 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 5, 2024-04-25T01:47:56.723 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-04-25T01:47:56.723 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-04-25T01:47:56.723 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-04-25T01:47:56.723 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-04-25T01:47:56.723 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-04-25T01:47:56.723 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-04-25T01:47:56.723 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-04-25T01:47:56.723 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-04-25T01:47:56.723 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-04-25T01:47:56.723 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-04-25T01:47:56.723 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4195732, 2024-04-25T01:47:56.723 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 4195424, 2024-04-25T01:47:56.723 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-04-25T01:47:56.724 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.724 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.724 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.724 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 5 2024-04-25T01:47:56.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.724 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-04-25T01:47:56.724 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 3, 2024-04-25T01:47:56.724 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 3, 2024-04-25T01:47:56.724 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 0, 2024-04-25T01:47:56.724 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 1, 2024-04-25T01:47:56.724 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 1, 2024-04-25T01:47:56.724 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 0, 2024-04-25T01:47:56.725 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 0, 2024-04-25T01:47:56.725 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 0, 2024-04-25T01:47:56.725 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 0, 2024-04-25T01:47:56.725 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-04-25T01:47:56.725 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-04-25T01:47:56.725 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-04-25T01:47:56.725 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 50308, 2024-04-25T01:47:56.725 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215420 2024-04-25T01:47:56.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.725 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-04-25T01:47:56.725 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-04-25T01:47:56.725 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-04-25T01:47:56.725 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-04-25T01:47:56.725 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-04-25T01:47:56.726 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-04-25T01:47:56.726 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-04-25T01:47:56.726 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.726 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.726 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.726 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-04-25T01:47:56.726 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.726 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.726 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.726 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-04-25T01:47:56.726 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-04-25T01:47:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-04-25T01:47:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.728 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-04-25T01:47:56.728 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.728 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.728 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.728 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-04-25T01:47:56.728 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.728 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.728 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.728 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-04-25T01:47:56.728 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.728 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.728 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.729 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-04-25T01:47:56.729 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.729 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.729 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.729 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-04-25T01:47:56.729 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.729 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.729 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.729 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.729 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-04-25T01:47:56.729 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.729 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.730 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.730 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-04-25T01:47:56.730 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.730 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.731 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.731 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-04-25T01:47:56.731 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.731 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.731 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.731 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-04-25T01:47:56.731 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.731 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.731 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.731 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-04-25T01:47:56.732 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.732 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.732 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.732 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-04-25T01:47:56.732 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.732 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.732 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.732 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-04-25T01:47:56.732 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.732 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.732 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.732 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.733 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-04-25T01:47:56.733 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.733 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.733 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.733 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.733 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-04-25T01:47:56.733 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.733 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.733 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.733 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.733 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-04-25T01:47:56.733 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.733 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.734 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.734 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-04-25T01:47:56.734 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.734 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.734 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.734 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-04-25T01:47:56.734 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.734 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.734 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.734 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-04-25T01:47:56.735 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.735 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.735 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.735 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-04-25T01:47:56.735 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.735 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.735 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.735 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-04-25T01:47:56.735 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.735 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.735 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.736 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-04-25T01:47:56.736 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.736 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.736 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.736 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-04-25T01:47:56.736 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.736 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.736 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.736 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-04-25T01:47:56.736 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.736 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.737 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.737 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.737 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-04-25T01:47:56.737 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.737 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.737 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.737 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.737 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:47:56.737 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:47:56.737 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-04-25T01:47:56.737 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.737 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.737 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.737 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:56.738 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.738 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-04-25T01:47:56.738 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-04-25T01:47:56.738 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 2, 2024-04-25T01:47:56.738 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 1, 2024-04-25T01:47:56.738 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-04-25T01:47:56.738 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-04-25T01:47:56.738 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-04-25T01:47:56.738 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-04-25T01:47:56.738 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 1320, 2024-04-25T01:47:56.738 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-04-25T01:47:56.738 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.738 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-04-25T01:47:56.739 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:47:56.739 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-04-25T01:47:56.739 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:47:56.739 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:47:56.739 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:47:56.739 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:47:56.739 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:47:56.739 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:47:56.739 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:47:56.739 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:47:56.739 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:47:56.739 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:47:56.739 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:47:56.740 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:47:56.740 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:47:56.740 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-04-25T01:47:56.740 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:47:56.740 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-04-25T01:47:56.740 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:47:56.740 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:47:56.740 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:47:56.740 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:47:56.740 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:47:56.740 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:47:56.740 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:47:56.740 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-04-25T01:47:56.740 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:47:56.741 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:47:56.741 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:47:56.741 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-04-25T01:47:56.741 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-04-25T01:47:56.741 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-04-25T01:47:56.741 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:47:56.741 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:47:56.741 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:47:56.741 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:47:56.741 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 189239, 2024-04-25T01:47:56.741 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 48, 2024-04-25T01:47:56.741 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:47:56.741 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-04-25T01:47:56.741 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-04-25T01:47:56.742 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-04-25T01:47:56.742 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:47:56.742 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-04-25T01:47:56.742 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:47:56.742 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-04-25T01:47:56.742 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9096, 2024-04-25T01:47:56.742 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 176, 2024-04-25T01:47:56.742 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:47:56.742 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:47:56.742 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-04-25T01:47:56.742 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-04-25T01:47:56.742 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 8464, 2024-04-25T01:47:56.742 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 13, 2024-04-25T01:47:56.743 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-04-25T01:47:56.743 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-04-25T01:47:56.743 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-04-25T01:47:56.743 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-04-25T01:47:56.743 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.743 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-04-25T01:47:56.743 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-04-25T01:47:56.743 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-04-25T01:47:56.743 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 706, 2024-04-25T01:47:56.743 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 21, 2024-04-25T01:47:56.743 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 3, 2024-04-25T01:47:56.743 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 706, 2024-04-25T01:47:56.743 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-04-25T01:47:56.743 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 706, 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.362991832, 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000514152 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 706, 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 707 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 706, 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 706, 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 443, 2024-04-25T01:47:56.744 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-04-25T01:47:56.745 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 261, 2024-04-25T01:47:56.745 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-04-25T01:47:56.745 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-04-25T01:47:56.745 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-04-25T01:47:56.745 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-04-25T01:47:56.745 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-04-25T01:47:56.745 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-04-25T01:47:56.745 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-04-25T01:47:56.745 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-04-25T01:47:56.745 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-04-25T01:47:56.745 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-04-25T01:47:56.745 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 1, 2024-04-25T01:47:56.745 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-04-25T01:47:56.745 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:47:56.746 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-04-25T01:47:56.746 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:47:56.746 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-04-25T01:47:56.746 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-04-25T01:47:56.746 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-04-25T01:47:56.746 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:47:56.746 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-04-25T01:47:56.746 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:47:56.746 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-04-25T01:47:56.746 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-04-25T01:47:56.746 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-04-25T01:47:56.746 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-04-25T01:47:56.746 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 94, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-04-25T01:47:56.747 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 4, 2024-04-25T01:47:56.748 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 5, 2024-04-25T01:47:56.748 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 5, 2024-04-25T01:47:56.748 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-04-25T01:47:56.748 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-04-25T01:47:56.748 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-04-25T01:47:56.748 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-04-25T01:47:56.748 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-04-25T01:47:56.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.748 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-04-25T01:47:56.748 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-04-25T01:47:56.748 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:47:56.748 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-04-25T01:47:56.748 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-04-25T01:47:56.748 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:56.749 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-25T01:47:56.750 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:56.750 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 812, 2024-04-25T01:47:56.750 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 891621, 2024-04-25T01:47:56.750 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:56.750 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 812, 2024-04-25T01:47:56.750 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-25T01:47:56.750 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-25T01:47:56.750 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 812, 2024-04-25T01:47:56.750 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 891621, 2024-04-25T01:47:56.750 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:56.750 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.750 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.750 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.750 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:56.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.751 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-04-25T01:47:56.751 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:56.751 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-25T01:47:56.751 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:56.751 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-25T01:47:56.751 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-25T01:47:56.751 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:56.751 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:56.751 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 706, 2024-04-25T01:47:56.751 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 1433, 2024-04-25T01:47:56.751 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 706, 2024-04-25T01:47:56.751 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1433, 2024-04-25T01:47:56.751 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:56.752 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.752 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.752 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.752 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:56.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.752 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-04-25T01:47:56.752 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:56.752 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-04-25T01:47:56.752 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:56.752 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-25T01:47:56.752 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-25T01:47:56.752 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:56.752 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:56.752 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 706, 2024-04-25T01:47:56.753 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 706, 2024-04-25T01:47:56.753 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 706, 2024-04-25T01:47:56.753 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 706, 2024-04-25T01:47:56.753 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:56.753 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.753 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.753 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.753 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:56.753 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.753 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:47:56.753 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:56.753 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-25T01:47:56.753 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:56.753 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-25T01:47:56.754 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-25T01:47:56.754 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:56.754 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:56.754 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-25T01:47:56.754 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-25T01:47:56.754 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-25T01:47:56.754 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-25T01:47:56.754 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:56.754 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.754 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.754 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.754 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:56.754 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:56.754 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x5618126846e0": { 2024-04-25T01:47:56.755 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:56.755 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-25T01:47:56.755 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:56.755 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-25T01:47:56.755 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-25T01:47:56.755 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:56.755 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:56.755 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-25T01:47:56.755 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-25T01:47:56.755 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-25T01:47:56.755 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-25T01:47:56.755 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:56.755 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:56.755 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:56.756 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:56.756 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:56.756 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:56.756 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-25T01:47:57.622 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T01:47:57.622 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T01:47:57.622 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T01:47:57.622 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-25T01:47:57.622 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T01:47:57.622 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T01:47:57.858 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:47:58.288 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:47:58 smithi196 ceph-mon[32978]: pgmap v434: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T01:47:58.302 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:47:58 smithi152 ceph-mon[32465]: pgmap v434: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T01:47:58.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:47:58 smithi078 ceph-mon[29469]: pgmap v434: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T01:47:59.640 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-25T01:47:59.640 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:47:59.640 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 32, 2024-04-25T01:47:59.640 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 36, 2024-04-25T01:47:59.640 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 7874, 2024-04-25T01:47:59.640 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 59942, 2024-04-25T01:47:59.640 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 14, 2024-04-25T01:47:59.641 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 7, 2024-04-25T01:47:59.641 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.007674601, 2024-04-25T01:47:59.641 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.001248579, 2024-04-25T01:47:59.641 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.004415533, 2024-04-25T01:47:59.641 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000416046, 2024-04-25T01:47:59.641 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:59.641 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 32, 2024-04-25T01:47:59.641 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.021968924, 2024-04-25T01:47:59.642 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000686528 2024-04-25T01:47:59.642 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.642 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:59.642 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 16, 2024-04-25T01:47:59.642 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000001956, 2024-04-25T01:47:59.642 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000122 2024-04-25T01:47:59.642 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.643 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1712, 2024-04-25T01:47:59.643 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 1696 2024-04-25T01:47:59.643 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.643 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:47:59.643 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 148, 2024-04-25T01:47:59.643 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 134, 2024-04-25T01:47:59.643 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 780286, 2024-04-25T01:47:59.644 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 68360, 2024-04-25T01:47:59.644 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 5, 2024-04-25T01:47:59.644 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-04-25T01:47:59.644 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.022053142, 2024-04-25T01:47:59.644 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.006170536, 2024-04-25T01:47:59.645 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.018626751, 2024-04-25T01:47:59.645 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.002452388, 2024-04-25T01:47:59.645 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:59.645 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 134, 2024-04-25T01:47:59.645 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.018086909, 2024-04-25T01:47:59.645 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000134976 2024-04-25T01:47:59.645 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.646 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:59.646 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.646 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.646 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.646 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.647 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 776496, 2024-04-25T01:47:59.647 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 56256 2024-04-25T01:47:59.647 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.647 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:47:59.647 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 697, 2024-04-25T01:47:59.647 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 508, 2024-04-25T01:47:59.647 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 220087, 2024-04-25T01:47:59.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 312228, 2024-04-25T01:47:59.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-04-25T01:47:59.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-04-25T01:47:59.648 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.054276979, 2024-04-25T01:47:59.649 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.021122071, 2024-04-25T01:47:59.649 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.039619537, 2024-04-25T01:47:59.649 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.001747199, 2024-04-25T01:47:59.649 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:47:59.649 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 508, 2024-04-25T01:47:59.649 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.017887374, 2024-04-25T01:47:59.649 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000035211 2024-04-25T01:47:59.650 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.650 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:47:59.650 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1115, 2024-04-25T01:47:59.650 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000219191, 2024-04-25T01:47:59.650 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000196 2024-04-25T01:47:59.651 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.651 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:47:59.651 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 182064 2024-04-25T01:47:59.651 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.651 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-04-25T01:47:59.651 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-04-25T01:47:59.651 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-04-25T01:47:59.652 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.652 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-04-25T01:47:59.652 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-25T01:47:59.652 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-25T01:47:59.652 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 50, 2024-04-25T01:47:59.652 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001326437, 2024-04-25T01:47:59.652 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000026528 2024-04-25T01:47:59.652 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:59.652 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.652 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-04-25T01:47:59.652 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-25T01:47:59.652 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-25T01:47:59.652 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-04-25T01:47:59.652 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000006571, 2024-04-25T01:47:59.653 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000003285 2024-04-25T01:47:59.653 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:59.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.653 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-04-25T01:47:59.653 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 1, 2024-04-25T01:47:59.653 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 1, 2024-04-25T01:47:59.653 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-04-25T01:47:59.653 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-04-25T01:47:59.653 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000103997, 2024-04-25T01:47:59.653 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000103997 2024-04-25T01:47:59.653 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.653 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-04-25T01:47:59.653 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-04-25T01:47:59.654 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-04-25T01:47:59.654 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-04-25T01:47:59.654 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-04-25T01:47:59.654 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-04-25T01:47:59.654 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-04-25T01:47:59.654 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 17, 2024-04-25T01:47:59.654 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 1, 2024-04-25T01:47:59.654 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 16, 2024-04-25T01:47:59.654 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-04-25T01:47:59.654 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 13, 2024-04-25T01:47:59.654 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 8, 2024-04-25T01:47:59.654 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 1, 2024-04-25T01:47:59.655 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 1, 2024-04-25T01:47:59.655 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 3, 2024-04-25T01:47:59.655 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 9, 2024-04-25T01:47:59.655 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 6, 2024-04-25T01:47:59.655 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-04-25T01:47:59.655 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 3, 2024-04-25T01:47:59.655 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:47:59.655 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-04-25T01:47:59.655 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-04-25T01:47:59.655 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-04-25T01:47:59.655 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-04-25T01:47:59.655 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-04-25T01:47:59.655 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-04-25T01:47:59.655 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-04-25T01:47:59.656 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-04-25T01:47:59.656 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-04-25T01:47:59.656 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:47:59.656 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-04-25T01:47:59.656 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 4, 2024-04-25T01:47:59.656 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-04-25T01:47:59.656 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-04-25T01:47:59.656 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:47:59.656 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-04-25T01:47:59.656 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:47:59.656 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:47:59.656 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:47:59.656 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:47:59.657 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:47:59.657 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-04-25T01:47:59.657 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-04-25T01:47:59.657 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T01:47:59.657 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-04-25T01:47:59.657 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:47:59.657 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:47:59.657 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:47:59.657 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-04-25T01:47:59.657 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-04-25T01:47:59.657 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-04-25T01:47:59.657 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-04-25T01:47:59.657 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.658 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-04-25T01:47:59.658 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-04-25T01:47:59.658 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 15, 2024-04-25T01:47:59.658 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 9, 2024-04-25T01:47:59.658 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 10, 2024-04-25T01:47:59.658 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-04-25T01:47:59.658 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-04-25T01:47:59.658 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-04-25T01:47:59.658 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:47:59.658 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-04-25T01:47:59.658 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-04-25T01:47:59.658 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-04-25T01:47:59.659 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-04-25T01:47:59.659 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-04-25T01:47:59.659 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:47:59.659 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:47:59.659 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-04-25T01:47:59.659 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-04-25T01:47:59.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:47:59.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:47:59.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:47:59.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-04-25T01:47:59.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-04-25T01:47:59.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:47:59.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-04-25T01:47:59.659 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-04-25T01:47:59.660 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.660 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-04-25T01:47:59.660 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-04-25T01:47:59.660 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 5, 2024-04-25T01:47:59.660 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-04-25T01:47:59.660 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-04-25T01:47:59.660 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 5, 2024-04-25T01:47:59.660 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-04-25T01:47:59.660 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-04-25T01:47:59.660 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 2, 2024-04-25T01:47:59.660 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-04-25T01:47:59.660 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-04-25T01:47:59.660 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-04-25T01:47:59.660 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4195732, 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 4, 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.007471192, 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001867798 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 21, 2024-04-25T01:47:59.661 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 27, 2024-04-25T01:47:59.662 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 6, 2024-04-25T01:47:59.662 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 20, 2024-04-25T01:47:59.662 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 25, 2024-04-25T01:47:59.662 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 5, 2024-04-25T01:47:59.662 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 17, 2024-04-25T01:47:59.662 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 25, 2024-04-25T01:47:59.662 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 8, 2024-04-25T01:47:59.662 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 1, 2024-04-25T01:47:59.662 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 1, 2024-04-25T01:47:59.662 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-04-25T01:47:59.662 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 54592, 2024-04-25T01:47:59.662 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223612 2024-04-25T01:47:59.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.663 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-04-25T01:47:59.663 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 1, 2024-04-25T01:47:59.663 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-04-25T01:47:59.663 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 1, 2024-04-25T01:47:59.663 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 6, 2024-04-25T01:47:59.663 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-04-25T01:47:59.663 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-04-25T01:47:59.663 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.663 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.663 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.663 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-04-25T01:47:59.663 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-04-25T01:47:59.664 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000103997, 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000103997 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-04-25T01:47:59.665 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.666 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.666 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.666 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-04-25T01:47:59.666 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.666 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.666 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.666 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.666 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-04-25T01:47:59.666 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.666 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.667 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.667 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-04-25T01:47:59.667 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.667 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.667 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.667 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.667 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-04-25T01:47:59.667 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.668 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.668 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.668 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-04-25T01:47:59.668 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.668 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.668 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.668 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.668 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-04-25T01:47:59.668 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.668 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-04-25T01:47:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-04-25T01:47:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-04-25T01:47:59.669 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.670 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.670 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-04-25T01:47:59.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.670 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.670 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-04-25T01:47:59.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.670 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.670 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.670 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.670 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-04-25T01:47:59.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.671 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.671 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-04-25T01:47:59.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.671 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.671 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-04-25T01:47:59.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.671 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.671 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.671 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.672 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-04-25T01:47:59.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.672 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.672 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-04-25T01:47:59.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.672 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.672 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.672 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-04-25T01:47:59.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.672 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.672 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.673 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.674 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-04-25T01:47:59.674 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.674 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.674 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.674 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.674 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:47:59.674 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:47:59.674 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-04-25T01:47:59.674 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.674 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.674 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.674 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:59.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.675 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-04-25T01:47:59.675 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-04-25T01:47:59.675 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 2, 2024-04-25T01:47:59.675 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 1, 2024-04-25T01:47:59.675 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-04-25T01:47:59.675 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-04-25T01:47:59.675 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 1, 2024-04-25T01:47:59.675 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 1, 2024-04-25T01:47:59.675 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 138, 2024-04-25T01:47:59.675 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-04-25T01:47:59.675 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.675 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-04-25T01:47:59.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:47:59.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-04-25T01:47:59.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:47:59.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:47:59.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:47:59.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:47:59.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:47:59.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:47:59.676 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:47:59.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:47:59.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:47:59.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:47:59.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:47:59.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:47:59.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:47:59.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-04-25T01:47:59.677 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:47:59.678 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-04-25T01:47:59.678 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:47:59.678 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:47:59.678 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:47:59.678 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:47:59.678 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:47:59.678 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:47:59.678 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:47:59.678 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-04-25T01:47:59.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:47:59.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:47:59.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:47:59.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-04-25T01:47:59.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-04-25T01:47:59.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-04-25T01:47:59.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:47:59.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:47:59.679 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:47:59.680 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:47:59.680 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 230279, 2024-04-25T01:47:59.680 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 73, 2024-04-25T01:47:59.680 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:47:59.680 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-04-25T01:47:59.680 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-04-25T01:47:59.680 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-04-25T01:47:59.680 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:47:59.681 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-04-25T01:47:59.681 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:47:59.681 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-04-25T01:47:59.681 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9096, 2024-04-25T01:47:59.681 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 176, 2024-04-25T01:47:59.681 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:47:59.681 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:47:59.681 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-04-25T01:47:59.681 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-04-25T01:47:59.682 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 85133, 2024-04-25T01:47:59.682 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 335, 2024-04-25T01:47:59.682 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-04-25T01:47:59.682 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-04-25T01:47:59.682 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-04-25T01:47:59.682 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-04-25T01:47:59.682 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.682 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-04-25T01:47:59.682 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-04-25T01:47:59.683 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-04-25T01:47:59.683 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 40, 2024-04-25T01:47:59.683 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 11730, 2024-04-25T01:47:59.683 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 28, 2024-04-25T01:47:59.683 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 40, 2024-04-25T01:47:59.683 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-04-25T01:47:59.683 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 40, 2024-04-25T01:47:59.683 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.218995079, 2024-04-25T01:47:59.684 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.005474876 2024-04-25T01:47:59.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.684 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-04-25T01:47:59.684 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-04-25T01:47:59.684 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 40, 2024-04-25T01:47:59.684 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 62 2024-04-25T01:47:59.684 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.684 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 40, 2024-04-25T01:47:59.685 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 0, 2024-04-25T01:47:59.685 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 40, 2024-04-25T01:47:59.685 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-04-25T01:47:59.685 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-04-25T01:47:59.685 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-04-25T01:47:59.685 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-04-25T01:47:59.685 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 0, 2024-04-25T01:47:59.685 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 4, 2024-04-25T01:47:59.685 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 7, 2024-04-25T01:47:59.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-04-25T01:47:59.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-04-25T01:47:59.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-04-25T01:47:59.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-04-25T01:47:59.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 6, 2024-04-25T01:47:59.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-04-25T01:47:59.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-04-25T01:47:59.686 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-04-25T01:47:59.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-04-25T01:47:59.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-04-25T01:47:59.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:47:59.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-04-25T01:47:59.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:47:59.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-04-25T01:47:59.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-04-25T01:47:59.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-04-25T01:47:59.687 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:47:59.688 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-04-25T01:47:59.688 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:47:59.688 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-04-25T01:47:59.688 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-04-25T01:47:59.688 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-04-25T01:47:59.688 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-04-25T01:47:59.688 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-04-25T01:47:59.688 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-04-25T01:47:59.688 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-04-25T01:47:59.689 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-04-25T01:47:59.689 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-04-25T01:47:59.689 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-04-25T01:47:59.689 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-04-25T01:47:59.689 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-04-25T01:47:59.689 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-04-25T01:47:59.689 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-04-25T01:47:59.689 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-04-25T01:47:59.690 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-04-25T01:47:59.690 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-04-25T01:47:59.690 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 94, 2024-04-25T01:47:59.690 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-04-25T01:47:59.690 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 4, 2024-04-25T01:47:59.690 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 8, 2024-04-25T01:47:59.690 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 8, 2024-04-25T01:47:59.690 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-04-25T01:47:59.690 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-04-25T01:47:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 14, 2024-04-25T01:47:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-04-25T01:47:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 1 2024-04-25T01:47:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-04-25T01:47:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-04-25T01:47:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:47:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-04-25T01:47:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-04-25T01:47:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-04-25T01:47:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-04-25T01:47:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.691 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-04-25T01:47:59.692 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-04-25T01:47:59.692 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T01:47:59.692 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-04-25T01:47:59.692 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-04-25T01:47:59.692 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-04-25T01:47:59.692 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-04-25T01:47:59.692 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-04-25T01:47:59.692 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.692 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:47:59.692 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:59.692 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-25T01:47:59.692 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:59.692 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 873, 2024-04-25T01:47:59.692 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 919870, 2024-04-25T01:47:59.693 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:59.693 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 873, 2024-04-25T01:47:59.693 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-25T01:47:59.693 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-25T01:47:59.693 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 873, 2024-04-25T01:47:59.693 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 919870, 2024-04-25T01:47:59.693 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:59.693 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.694 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.694 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.694 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:59.694 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.694 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-04-25T01:47:59.694 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:59.694 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-25T01:47:59.694 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:59.694 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-25T01:47:59.694 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-25T01:47:59.694 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:59.694 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:59.694 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 40, 2024-04-25T01:47:59.695 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 11730, 2024-04-25T01:47:59.695 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 40, 2024-04-25T01:47:59.695 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 11730, 2024-04-25T01:47:59.695 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:59.695 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.695 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.695 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.695 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:59.695 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.695 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-04-25T01:47:59.695 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:59.695 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-04-25T01:47:59.695 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 40, 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 40, 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 40, 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 40, 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.696 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:47:59.697 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:59.697 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-25T01:47:59.697 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:59.697 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-25T01:47:59.697 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-25T01:47:59.697 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:59.697 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-25T01:47:59.697 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-25T01:47:59.697 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-25T01:47:59.697 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-25T01:47:59.697 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-25T01:47:59.697 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:59.697 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.698 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.698 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.698 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:59.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:47:59.698 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55e891b30500": { 2024-04-25T01:47:59.698 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:47:59.698 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-25T01:47:59.698 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:47:59.698 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 5, 2024-04-25T01:47:59.698 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 1428, 2024-04-25T01:47:59.698 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:47:59.698 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 5, 2024-04-25T01:47:59.698 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-25T01:47:59.698 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-25T01:47:59.699 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 4, 2024-04-25T01:47:59.699 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1428, 2024-04-25T01:47:59.699 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:47:59.699 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:47:59.699 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:47:59.699 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:47:59.699 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:59.699 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:47:59.699 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-25T01:48:00.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:00 smithi152 ceph-mon[32465]: pgmap v435: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T01:48:00.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:00 smithi196 ceph-mon[32978]: pgmap v435: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T01:48:00.482 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T01:48:00.482 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T01:48:00.482 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-25T01:48:00.482 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=15 2024-04-25T01:48:00.482 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T01:48:00.482 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid df36bea8-02a3-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T01:48:00.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:00 smithi078 ceph-mon[29469]: pgmap v435: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T01:48:00.719 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T01:48:02.247 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:02 smithi196 ceph-mon[32978]: pgmap v436: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T01:48:02.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:02 smithi152 ceph-mon[32465]: pgmap v436: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T01:48:02.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:02 smithi078 ceph-mon[29469]: pgmap v436: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T01:48:02.694 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-25T01:48:02.695 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T01:48:02.695 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 688, 2024-04-25T01:48:02.695 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 440, 2024-04-25T01:48:02.695 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 213084, 2024-04-25T01:48:02.695 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 101686, 2024-04-25T01:48:02.695 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 8, 2024-04-25T01:48:02.695 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 2, 2024-04-25T01:48:02.695 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.053289374, 2024-04-25T01:48:02.695 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.018997402, 2024-04-25T01:48:02.695 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.039799579, 2024-04-25T01:48:02.695 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.001916372, 2024-04-25T01:48:02.695 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:48:02.696 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 442, 2024-04-25T01:48:02.696 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.023146412, 2024-04-25T01:48:02.696 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000052367 2024-04-25T01:48:02.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.696 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:48:02.696 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1086, 2024-04-25T01:48:02.696 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000212298, 2024-04-25T01:48:02.696 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000195 2024-04-25T01:48:02.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.696 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T01:48:02.696 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T01:48:02.696 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.696 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T01:48:02.697 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 158, 2024-04-25T01:48:02.697 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 152, 2024-04-25T01:48:02.697 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 766057, 2024-04-25T01:48:02.697 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 118967, 2024-04-25T01:48:02.697 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 11, 2024-04-25T01:48:02.697 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 8, 2024-04-25T01:48:02.697 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.024210482, 2024-04-25T01:48:02.697 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.006441505, 2024-04-25T01:48:02.697 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.020077200, 2024-04-25T01:48:02.697 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.002870815, 2024-04-25T01:48:02.697 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:48:02.697 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 148, 2024-04-25T01:48:02.697 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.014877393, 2024-04-25T01:48:02.698 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000100522 2024-04-25T01:48:02.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.698 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:48:02.698 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 11, 2024-04-25T01:48:02.698 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000001247, 2024-04-25T01:48:02.698 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000113 2024-04-25T01:48:02.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.698 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 762976, 2024-04-25T01:48:02.698 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 61632 2024-04-25T01:48:02.698 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.698 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T01:48:02.698 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 16, 2024-04-25T01:48:02.698 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 83, 2024-04-25T01:48:02.699 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 4241, 2024-04-25T01:48:02.699 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 190309, 2024-04-25T01:48:02.699 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 6, 2024-04-25T01:48:02.699 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-04-25T01:48:02.699 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.006935699, 2024-04-25T01:48:02.699 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.003029116, 2024-04-25T01:48:02.699 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.002553330, 2024-04-25T01:48:02.699 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000493405, 2024-04-25T01:48:02.699 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T01:48:02.699 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 81, 2024-04-25T01:48:02.699 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.023194263, 2024-04-25T01:48:02.699 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000286348 2024-04-25T01:48:02.699 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.700 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-25T01:48:02.700 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.700 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.700 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.700 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T01:48:02.700 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 182944 2024-04-25T01:48:02.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.700 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-04-25T01:48:02.700 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-04-25T01:48:02.700 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-04-25T01:48:02.700 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.700 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-04-25T01:48:02.701 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-25T01:48:02.701 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-25T01:48:02.701 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 43, 2024-04-25T01:48:02.701 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002067860, 2024-04-25T01:48:02.701 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000048089 2024-04-25T01:48:02.701 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:48:02.701 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.701 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-04-25T01:48:02.701 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-25T01:48:02.701 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-25T01:48:02.701 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-04-25T01:48:02.701 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000010988, 2024-04-25T01:48:02.701 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000005494 2024-04-25T01:48:02.701 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:48:02.702 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.702 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-04-25T01:48:02.702 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-04-25T01:48:02.702 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-04-25T01:48:02.702 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-04-25T01:48:02.702 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.702 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.702 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.702 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.703 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-04-25T01:48:02.703 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-04-25T01:48:02.703 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-04-25T01:48:02.703 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-04-25T01:48:02.703 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-04-25T01:48:02.703 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-04-25T01:48:02.703 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-04-25T01:48:02.703 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 16, 2024-04-25T01:48:02.703 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-04-25T01:48:02.703 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 16, 2024-04-25T01:48:02.703 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-04-25T01:48:02.703 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 13, 2024-04-25T01:48:02.703 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 8, 2024-04-25T01:48:02.703 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 3, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 6, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 4, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 2, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-04-25T01:48:02.704 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-04-25T01:48:02.705 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-04-25T01:48:02.705 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-04-25T01:48:02.705 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-04-25T01:48:02.705 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 4, 2024-04-25T01:48:02.705 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-04-25T01:48:02.705 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-04-25T01:48:02.705 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T01:48:02.705 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-04-25T01:48:02.705 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T01:48:02.705 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-04-25T01:48:02.705 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-04-25T01:48:02.705 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T01:48:02.705 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T01:48:02.706 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-04-25T01:48:02.706 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-04-25T01:48:02.706 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T01:48:02.706 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-04-25T01:48:02.706 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T01:48:02.706 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T01:48:02.706 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T01:48:02.706 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-04-25T01:48:02.706 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-04-25T01:48:02.706 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-04-25T01:48:02.706 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-04-25T01:48:02.706 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.706 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 10, 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 6, 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 7, 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-04-25T01:48:02.707 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 4, 2024-04-25T01:48:02.708 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-04-25T01:48:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-04-25T01:48:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 4, 2024-04-25T01:48:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-04-25T01:48:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-04-25T01:48:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 2, 2024-04-25T01:48:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-04-25T01:48:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-04-25T01:48:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-04-25T01:48:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-04-25T01:48:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-04-25T01:48:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-04-25T01:48:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-04-25T01:48:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4195424, 2024-04-25T01:48:02.709 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-04-25T01:48:02.710 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-04-25T01:48:02.710 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3, 2024-04-25T01:48:02.710 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.006074957, 2024-04-25T01:48:02.710 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002024985 2024-04-25T01:48:02.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.710 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-04-25T01:48:02.710 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.710 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-04-25T01:48:02.710 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 20, 2024-04-25T01:48:02.710 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 26, 2024-04-25T01:48:02.710 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 6, 2024-04-25T01:48:02.710 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 19, 2024-04-25T01:48:02.710 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 24, 2024-04-25T01:48:02.710 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 5, 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 16, 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 24, 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 8, 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 51936, 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223612 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 4, 2024-04-25T01:48:02.711 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.712 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.713 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-04-25T01:48:02.713 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.713 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.713 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.713 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-04-25T01:48:02.713 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.713 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.713 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.713 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.713 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-04-25T01:48:02.713 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.713 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.714 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.714 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-04-25T01:48:02.714 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.714 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.714 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.714 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-04-25T01:48:02.714 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.714 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.714 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.714 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.714 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-04-25T01:48:02.714 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.715 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.715 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.715 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-04-25T01:48:02.715 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.715 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.715 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.715 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.715 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-04-25T01:48:02.715 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.716 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.716 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.716 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.716 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-04-25T01:48:02.716 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.717 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.717 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.717 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-04-25T01:48:02.717 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.717 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.717 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.717 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.717 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-04-25T01:48:02.717 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.717 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.717 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.718 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-04-25T01:48:02.718 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.718 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.718 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.718 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-04-25T01:48:02.718 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.718 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.718 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.718 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.718 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-04-25T01:48:02.719 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.719 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.719 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.719 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-04-25T01:48:02.719 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.719 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.719 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.719 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-04-25T01:48:02.719 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.719 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.719 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.719 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.720 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-04-25T01:48:02.720 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.720 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.720 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.720 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-04-25T01:48:02.720 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.720 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.720 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.720 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.720 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-04-25T01:48:02.720 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.720 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.720 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.721 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-04-25T01:48:02.721 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.721 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.721 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.721 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-04-25T01:48:02.721 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.721 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.721 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.721 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.722 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-04-25T01:48:02.722 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.722 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.722 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.722 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-04-25T01:48:02.723 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.723 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.723 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.723 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-04-25T01:48:02.723 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.723 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.723 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.723 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-04-25T01:48:02.723 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.723 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.723 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.724 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-04-25T01:48:02.724 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.724 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.724 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.724 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-04-25T01:48:02.724 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-04-25T01:48:02.724 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-04-25T01:48:02.724 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.724 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.724 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.724 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:48:02.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.725 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-04-25T01:48:02.725 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 0, 2024-04-25T01:48:02.725 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 1, 2024-04-25T01:48:02.725 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 1, 2024-04-25T01:48:02.725 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 0, 2024-04-25T01:48:02.725 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-04-25T01:48:02.725 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-04-25T01:48:02.725 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-04-25T01:48:02.725 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 65, 2024-04-25T01:48:02.725 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-04-25T01:48:02.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.725 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-04-25T01:48:02.726 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-04-25T01:48:02.726 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-04-25T01:48:02.726 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T01:48:02.726 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-04-25T01:48:02.726 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T01:48:02.726 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-04-25T01:48:02.726 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T01:48:02.726 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T01:48:02.726 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T01:48:02.726 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T01:48:02.726 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T01:48:02.726 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-04-25T01:48:02.726 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T01:48:02.727 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T01:48:02.727 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-04-25T01:48:02.727 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-04-25T01:48:02.727 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-04-25T01:48:02.727 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-04-25T01:48:02.727 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T01:48:02.727 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T01:48:02.727 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T01:48:02.727 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T01:48:02.727 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T01:48:02.727 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-04-25T01:48:02.727 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-04-25T01:48:02.727 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-04-25T01:48:02.727 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T01:48:02.728 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T01:48:02.728 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-04-25T01:48:02.728 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-04-25T01:48:02.728 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-04-25T01:48:02.728 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-04-25T01:48:02.728 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T01:48:02.728 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-04-25T01:48:02.728 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T01:48:02.728 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-04-25T01:48:02.728 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 230039, 2024-04-25T01:48:02.728 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 71, 2024-04-25T01:48:02.728 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-04-25T01:48:02.728 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-04-25T01:48:02.728 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-04-25T01:48:02.729 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-04-25T01:48:02.729 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-04-25T01:48:02.729 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-04-25T01:48:02.729 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-04-25T01:48:02.729 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-04-25T01:48:02.729 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 9096, 2024-04-25T01:48:02.729 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 176, 2024-04-25T01:48:02.729 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T01:48:02.729 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-04-25T01:48:02.729 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-04-25T01:48:02.729 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-04-25T01:48:02.729 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 80573, 2024-04-25T01:48:02.729 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 318, 2024-04-25T01:48:02.729 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-04-25T01:48:02.730 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-04-25T01:48:02.730 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-04-25T01:48:02.730 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-04-25T01:48:02.730 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.730 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-04-25T01:48:02.730 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-04-25T01:48:02.730 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-04-25T01:48:02.730 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 39, 2024-04-25T01:48:02.730 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 11422, 2024-04-25T01:48:02.730 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-04-25T01:48:02.730 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 39, 2024-04-25T01:48:02.730 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-04-25T01:48:02.730 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 39, 2024-04-25T01:48:02.731 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.212995226, 2024-04-25T01:48:02.731 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.005461416 2024-04-25T01:48:02.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.731 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-04-25T01:48:02.731 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-04-25T01:48:02.731 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 39, 2024-04-25T01:48:02.731 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 61 2024-04-25T01:48:02.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.731 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 39, 2024-04-25T01:48:02.731 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 0, 2024-04-25T01:48:02.731 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 39, 2024-04-25T01:48:02.731 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-04-25T01:48:02.731 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-04-25T01:48:02.732 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-04-25T01:48:02.732 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-04-25T01:48:02.732 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 0, 2024-04-25T01:48:02.732 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 3, 2024-04-25T01:48:02.732 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 7, 2024-04-25T01:48:02.732 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-04-25T01:48:02.732 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-04-25T01:48:02.732 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-04-25T01:48:02.732 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-04-25T01:48:02.732 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 6, 2024-04-25T01:48:02.732 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-04-25T01:48:02.732 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-04-25T01:48:02.732 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-04-25T01:48:02.732 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-04-25T01:48:02.733 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-04-25T01:48:02.733 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-04-25T01:48:02.733 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-04-25T01:48:02.733 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-04-25T01:48:02.733 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-04-25T01:48:02.733 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-04-25T01:48:02.733 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-04-25T01:48:02.733 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T01:48:02.733 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-04-25T01:48:02.733 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-04-25T01:48:02.733 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-04-25T01:48:02.733 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-04-25T01:48:02.733 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-04-25T01:48:02.733 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-04-25T01:48:02.734 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-04-25T01:48:02.734 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-04-25T01:48:02.734 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-04-25T01:48:02.734 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-04-25T01:48:02.734 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-04-25T01:48:02.734 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-04-25T01:48:02.734 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-04-25T01:48:02.734 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-04-25T01:48:02.734 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-04-25T01:48:02.734 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-04-25T01:48:02.734 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-04-25T01:48:02.734 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-04-25T01:48:02.734 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-04-25T01:48:02.734 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 94, 2024-04-25T01:48:02.735 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-04-25T01:48:02.735 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 4, 2024-04-25T01:48:02.735 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 9, 2024-04-25T01:48:02.735 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 9, 2024-04-25T01:48:02.735 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 0, 2024-04-25T01:48:02.735 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-04-25T01:48:02.735 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 14, 2024-04-25T01:48:02.735 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-04-25T01:48:02.735 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 1 2024-04-25T01:48:02.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.735 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-04-25T01:48:02.735 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-04-25T01:48:02.735 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-04-25T01:48:02.735 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-04-25T01:48:02.736 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-04-25T01:48:02.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.736 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-04-25T01:48:02.736 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-04-25T01:48:02.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.736 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-04-25T01:48:02.736 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-04-25T01:48:02.736 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T01:48:02.736 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-04-25T01:48:02.736 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-04-25T01:48:02.736 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-04-25T01:48:02.736 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-04-25T01:48:02.736 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-04-25T01:48:02.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.737 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T01:48:02.737 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:48:02.737 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-25T01:48:02.737 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:48:02.737 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 858, 2024-04-25T01:48:02.737 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 896127, 2024-04-25T01:48:02.737 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:48:02.737 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 858, 2024-04-25T01:48:02.737 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-25T01:48:02.737 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-25T01:48:02.737 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 858, 2024-04-25T01:48:02.737 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 896127, 2024-04-25T01:48:02.737 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:48:02.738 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.738 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.738 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.738 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:48:02.738 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.738 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-04-25T01:48:02.738 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:48:02.738 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-25T01:48:02.738 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:48:02.738 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-25T01:48:02.738 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-25T01:48:02.738 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:48:02.738 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-25T01:48:02.739 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 39, 2024-04-25T01:48:02.739 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 11422, 2024-04-25T01:48:02.739 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 39, 2024-04-25T01:48:02.739 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 11422, 2024-04-25T01:48:02.739 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:48:02.739 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.739 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.739 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.739 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:48:02.739 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.739 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-04-25T01:48:02.739 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:48:02.739 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-04-25T01:48:02.739 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:48:02.740 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-25T01:48:02.740 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-25T01:48:02.740 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:48:02.740 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-25T01:48:02.740 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 39, 2024-04-25T01:48:02.740 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 39, 2024-04-25T01:48:02.740 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 39, 2024-04-25T01:48:02.740 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 39, 2024-04-25T01:48:02.740 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:48:02.740 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.740 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.740 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.740 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:48:02.740 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.741 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-04-25T01:48:02.741 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:48:02.741 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-25T01:48:02.741 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:48:02.741 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-25T01:48:02.741 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-25T01:48:02.741 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:48:02.741 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-25T01:48:02.741 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-25T01:48:02.741 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-25T01:48:02.741 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-25T01:48:02.741 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-25T01:48:02.741 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:48:02.741 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x561cc7b86500": { 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 4, 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 1120, 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 4, 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-25T01:48:02.742 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3, 2024-04-25T01:48:02.743 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1120, 2024-04-25T01:48:02.743 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-25T01:48:02.743 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-25T01:48:02.743 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-25T01:48:02.743 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-25T01:48:02.743 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:48:02.743 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T01:48:02.743 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-25T01:48:03.517 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-25T01:48:03.517 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-25T01:48:03.517 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-25T01:48:03.517 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=10 2024-04-25T01:48:03.517 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T01:48:03.527 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T01:48:03.527 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:48:03.527 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:48:03.547 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-25T01:48:03.547 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:48:03.606 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:48:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.760+0000 7f3e2fd42640 1 -- 172.21.15.78:0/2491586510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e28075780 msgr2=0x7f3e28073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.760+0000 7f3e2fd42640 1 --2- 172.21.15.78:0/2491586510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e28075780 0x7f3e28073bd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009900 tx=0x7f3e2402a410 comp rx=0 tx=0).stop 2024-04-25T01:48:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.760+0000 7f3e2fd42640 1 -- 172.21.15.78:0/2491586510 shutdown_connections 2024-04-25T01:48:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.760+0000 7f3e2fd42640 1 --2- 172.21.15.78:0/2491586510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e28109650 0x7f3e2810bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.760+0000 7f3e2fd42640 1 --2- 172.21.15.78:0/2491586510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e280741a0 0x7f3e28074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.760+0000 7f3e2fd42640 1 --2- 172.21.15.78:0/2491586510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e28075780 0x7f3e28073bd0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.761+0000 7f3e2fd42640 1 -- 172.21.15.78:0/2491586510 >> 172.21.15.78:0/2491586510 conn(0x7f3e280fcd40 msgr2=0x7f3e280ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.761+0000 7f3e2fd42640 1 -- 172.21.15.78:0/2491586510 shutdown_connections 2024-04-25T01:48:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.761+0000 7f3e2fd42640 1 -- 172.21.15.78:0/2491586510 wait complete. 2024-04-25T01:48:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.761+0000 7f3e2fd42640 1 Processor -- start 2024-04-25T01:48:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.761+0000 7f3e2fd42640 1 -- start start 2024-04-25T01:48:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.761+0000 7f3e2fd42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e280741a0 0x7f3e28199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.761+0000 7f3e2fd42640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e28075780 0x7f3e2819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.761+0000 7f3e2fd42640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e28109650 0x7f3e281a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.761+0000 7f3e2fd42640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2810f5f0 con 0x7f3e280741a0 2024-04-25T01:48:03.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.761+0000 7f3e2fd42640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2810f470 con 0x7f3e28075780 2024-04-25T01:48:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.761+0000 7f3e2fd42640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2810f770 con 0x7f3e28109650 2024-04-25T01:48:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.761+0000 7f3e2dade640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e280741a0 0x7f3e28199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.761+0000 7f3e2dade640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e280741a0 0x7f3e28199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57050/0 (socket says 172.21.15.78:57050) 2024-04-25T01:48:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.761+0000 7f3e2dade640 1 -- 172.21.15.78:0/3963585946 learned_addr learned my addr 172.21.15.78:0/3963585946 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:48:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.762+0000 7f3e2e2df640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e28109650 0x7f3e281a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.762+0000 7f3e2d2dd640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e28075780 0x7f3e2819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.762+0000 7f3e2dade640 1 -- 172.21.15.78:0/3963585946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e28109650 msgr2=0x7f3e281a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.762+0000 7f3e2dade640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e28109650 0x7f3e281a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.762+0000 7f3e2dade640 1 -- 172.21.15.78:0/3963585946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e28075780 msgr2=0x7f3e2819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.762+0000 7f3e2dade640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e28075780 0x7f3e2819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.762+0000 7f3e2dade640 1 -- 172.21.15.78:0/3963585946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e281a18c0 con 0x7f3e280741a0 2024-04-25T01:48:03.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.762+0000 7f3e2d2dd640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e28075780 0x7f3e2819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:48:03.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.762+0000 7f3e2dade640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e280741a0 0x7f3e28199c80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009230 tx=0x7f3e24031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:03.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.762+0000 7f3e1effd640 1 -- 172.21.15.78:0/3963585946 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2403f030 con 0x7f3e280741a0 2024-04-25T01:48:03.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.762+0000 7f3e1effd640 1 -- 172.21.15.78:0/3963585946 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e24003ad0 con 0x7f3e280741a0 2024-04-25T01:48:03.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.762+0000 7f3e2fd42640 1 -- 172.21.15.78:0/3963585946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e281a1b50 con 0x7f3e280741a0 2024-04-25T01:48:03.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.762+0000 7f3e1effd640 1 -- 172.21.15.78:0/3963585946 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24035070 con 0x7f3e280741a0 2024-04-25T01:48:03.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.762+0000 7f3e2fd42640 1 -- 172.21.15.78:0/3963585946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e281a2040 con 0x7f3e280741a0 2024-04-25T01:48:03.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.763+0000 7f3e2fd42640 1 -- 172.21.15.78:0/3963585946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e28108380 con 0x7f3e280741a0 2024-04-25T01:48:03.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.764+0000 7f3e1effd640 1 -- 172.21.15.78:0/3963585946 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f3e24042030 con 0x7f3e280741a0 2024-04-25T01:48:03.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.765+0000 7f3e1effd640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3e140a2b90 0x7f3e140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:03.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.765+0000 7f3e2d2dd640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3e140a2b90 0x7f3e140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:03.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.765+0000 7f3e1effd640 1 -- 172.21.15.78:0/3963585946 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f3e240efe00 con 0x7f3e280741a0 2024-04-25T01:48:03.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.765+0000 7f3e2d2dd640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3e140a2b90 0x7f3e140a5050 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e2819b180 tx=0x7f3e18005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:03.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.768+0000 7f3e1effd640 1 -- 172.21.15.78:0/3963585946 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3e240bc980 con 0x7f3e280741a0 2024-04-25T01:48:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.945+0000 7f3e2fd42640 1 -- 172.21.15.78:0/3963585946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e2802d370 con 0x7f3e280741a0 2024-04-25T01:48:03.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.947+0000 7f3e1effd640 1 -- 172.21.15.78:0/3963585946 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24844 (secure 0 0 0) 0x7f3e240bc280 con 0x7f3e280741a0 2024-04-25T01:48:03.943 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:48:03.943 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":94,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:45:18.989934+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:48:03.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.949+0000 7f3e2fd42640 1 -- 172.21.15.78:0/3963585946 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3e140a2b90 msgr2=0x7f3e140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.949+0000 7f3e2fd42640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3e140a2b90 0x7f3e140a5050 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e2819b180 tx=0x7f3e18005f90 comp rx=0 tx=0).stop 2024-04-25T01:48:03.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.949+0000 7f3e2fd42640 1 -- 172.21.15.78:0/3963585946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e280741a0 msgr2=0x7f3e28199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.949+0000 7f3e2fd42640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e280741a0 0x7f3e28199c80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009230 tx=0x7f3e24031b60 comp rx=0 tx=0).stop 2024-04-25T01:48:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.949+0000 7f3e2fd42640 1 -- 172.21.15.78:0/3963585946 shutdown_connections 2024-04-25T01:48:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.949+0000 7f3e2fd42640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e28109650 0x7f3e281a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.949+0000 7f3e2fd42640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e28075780 0x7f3e2819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.949+0000 7f3e2fd42640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f3e140a2b90 0x7f3e140a5050 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.949+0000 7f3e2fd42640 1 --2- 172.21.15.78:0/3963585946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e280741a0 0x7f3e28199c80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.949+0000 7f3e2fd42640 1 -- 172.21.15.78:0/3963585946 >> 172.21.15.78:0/3963585946 conn(0x7f3e280fcd40 msgr2=0x7f3e28105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.949+0000 7f3e2fd42640 1 -- 172.21.15.78:0/3963585946 shutdown_connections 2024-04-25T01:48:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:03.949+0000 7f3e2fd42640 1 -- 172.21.15.78:0/3963585946 wait complete. 2024-04-25T01:48:03.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:38:39.136418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:43:15.613881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:43:16.600270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:43:19.650457+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:48:03.958 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:48:04.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.063+0000 7fb811190640 1 -- 172.21.15.78:0/3811502071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb80c101890 msgr2=0x7fb80c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.063+0000 7fb811190640 1 --2- 172.21.15.78:0/3811502071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb80c101890 0x7fb80c101d10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4009900 tx=0x7fb7f402a410 comp rx=0 tx=0).stop 2024-04-25T01:48:04.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.063+0000 7fb811190640 1 -- 172.21.15.78:0/3811502071 shutdown_connections 2024-04-25T01:48:04.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.063+0000 7fb811190640 1 --2- 172.21.15.78:0/3811502071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb80c102250 0x7fb80c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.063+0000 7fb811190640 1 --2- 172.21.15.78:0/3811502071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb80c101890 0x7fb80c101d10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.063+0000 7fb811190640 1 --2- 172.21.15.78:0/3811502071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80c1006a0 0x7fb80c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.063+0000 7fb811190640 1 -- 172.21.15.78:0/3811502071 >> 172.21.15.78:0/3811502071 conn(0x7fb80c0fbc30 msgr2=0x7fb80c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:04.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.063+0000 7fb811190640 1 -- 172.21.15.78:0/3811502071 shutdown_connections 2024-04-25T01:48:04.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.063+0000 7fb811190640 1 -- 172.21.15.78:0/3811502071 wait complete. 2024-04-25T01:48:04.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb811190640 1 Processor -- start 2024-04-25T01:48:04.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb811190640 1 -- start start 2024-04-25T01:48:04.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb811190640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80c1006a0 0x7fb80c196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:04.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb811190640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb80c101890 0x7fb80c196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:04.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb811190640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb80c102250 0x7fb80c19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:04.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb811190640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c10c3e0 con 0x7fb80c101890 2024-04-25T01:48:04.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb811190640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c10c260 con 0x7fb80c102250 2024-04-25T01:48:04.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb811190640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c10c560 con 0x7fb80c1006a0 2024-04-25T01:48:04.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb803fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb80c101890 0x7fb80c196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:04.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb803fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb80c101890 0x7fb80c196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57068/0 (socket says 172.21.15.78:57068) 2024-04-25T01:48:04.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb803fff640 1 -- 172.21.15.78:0/773413744 learned_addr learned my addr 172.21.15.78:0/773413744 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:48:04.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb80ad9d640 1 --2- 172.21.15.78:0/773413744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80c1006a0 0x7fb80c196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:04.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb80b59e640 1 --2- 172.21.15.78:0/773413744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb80c102250 0x7fb80c19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:04.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb803fff640 1 -- 172.21.15.78:0/773413744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80c1006a0 msgr2=0x7fb80c196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb803fff640 1 --2- 172.21.15.78:0/773413744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80c1006a0 0x7fb80c196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb803fff640 1 -- 172.21.15.78:0/773413744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb80c102250 msgr2=0x7fb80c19e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb803fff640 1 --2- 172.21.15.78:0/773413744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb80c102250 0x7fb80c19e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb803fff640 1 -- 172.21.15.78:0/773413744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c19e6a0 con 0x7fb80c101890 2024-04-25T01:48:04.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.064+0000 7fb803fff640 1 --2- 172.21.15.78:0/773413744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb80c101890 0x7fb80c196fc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb7f402aa00 tx=0x7fb7f4032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:04.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.065+0000 7fb808d99640 1 -- 172.21.15.78:0/773413744 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f403f030 con 0x7fb80c101890 2024-04-25T01:48:04.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.065+0000 7fb808d99640 1 -- 172.21.15.78:0/773413744 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f4003ce0 con 0x7fb80c101890 2024-04-25T01:48:04.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.065+0000 7fb811190640 1 -- 172.21.15.78:0/773413744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80c19e930 con 0x7fb80c101890 2024-04-25T01:48:04.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.065+0000 7fb808d99640 1 -- 172.21.15.78:0/773413744 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4035070 con 0x7fb80c101890 2024-04-25T01:48:04.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.065+0000 7fb811190640 1 -- 172.21.15.78:0/773413744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80c19eea0 con 0x7fb80c101890 2024-04-25T01:48:04.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.066+0000 7fb811190640 1 -- 172.21.15.78:0/773413744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb80c107a90 con 0x7fb80c101890 2024-04-25T01:48:04.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.067+0000 7fb808d99640 1 -- 172.21.15.78:0/773413744 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb7f4042030 con 0x7fb80c101890 2024-04-25T01:48:04.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.067+0000 7fb808d99640 1 --2- 172.21.15.78:0/773413744 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb7f80a2b90 0x7fb7f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:04.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.067+0000 7fb80ad9d640 1 --2- 172.21.15.78:0/773413744 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb7f80a2b90 0x7fb7f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:04.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.067+0000 7fb808d99640 1 -- 172.21.15.78:0/773413744 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fb7f40eff80 con 0x7fb80c101890 2024-04-25T01:48:04.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.068+0000 7fb80ad9d640 1 --2- 172.21.15.78:0/773413744 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb7f80a2b90 0x7fb7f80a5050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb80c1016f0 tx=0x7fb7fc009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:04.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.071+0000 7fb808d99640 1 -- 172.21.15.78:0/773413744 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb7f40bcb00 con 0x7fb80c101890 2024-04-25T01:48:04.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.244+0000 7fb811190640 1 -- 172.21.15.78:0/773413744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb80c02d3c0 con 0x7fb80c101890 2024-04-25T01:48:04.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.245+0000 7fb808d99640 1 -- 172.21.15.78:0/773413744 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fb7f40bc400 con 0x7fb80c101890 2024-04-25T01:48:04.241 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-04-25T01:48:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.246+0000 7fb811190640 1 -- 172.21.15.78:0/773413744 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb7f80a2b90 msgr2=0x7fb7f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.246+0000 7fb811190640 1 --2- 172.21.15.78:0/773413744 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb7f80a2b90 0x7fb7f80a5050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb80c1016f0 tx=0x7fb7fc009110 comp rx=0 tx=0).stop 2024-04-25T01:48:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.246+0000 7fb811190640 1 -- 172.21.15.78:0/773413744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb80c101890 msgr2=0x7fb80c196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.246+0000 7fb811190640 1 --2- 172.21.15.78:0/773413744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb80c101890 0x7fb80c196fc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb7f402aa00 tx=0x7fb7f4032da0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.246+0000 7fb811190640 1 -- 172.21.15.78:0/773413744 shutdown_connections 2024-04-25T01:48:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.246+0000 7fb811190640 1 --2- 172.21.15.78:0/773413744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb80c102250 0x7fb80c19e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.246+0000 7fb811190640 1 --2- 172.21.15.78:0/773413744 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb7f80a2b90 0x7fb7f80a5050 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.246+0000 7fb811190640 1 --2- 172.21.15.78:0/773413744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb80c101890 0x7fb80c196fc0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.246+0000 7fb811190640 1 --2- 172.21.15.78:0/773413744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80c1006a0 0x7fb80c196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.246+0000 7fb811190640 1 -- 172.21.15.78:0/773413744 >> 172.21.15.78:0/773413744 conn(0x7fb80c0fbc30 msgr2=0x7fb80c104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.247+0000 7fb811190640 1 -- 172.21.15.78:0/773413744 shutdown_connections 2024-04-25T01:48:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.247+0000 7fb811190640 1 -- 172.21.15.78:0/773413744 wait complete. 2024-04-25T01:48:04.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:04 smithi078 ceph-mon[29469]: pgmap v437: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T01:48:04.253 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:04 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3963585946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:48:04.254 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:48:04.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:04 smithi152 ceph-mon[32465]: pgmap v437: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T01:48:04.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:04 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3963585946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:48:04.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.359+0000 7fb8a84ba640 1 -- 172.21.15.78:0/398793670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8a00ffff0 msgr2=0x7fb8a0102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.359+0000 7fb8a84ba640 1 --2- 172.21.15.78:0/398793670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8a00ffff0 0x7fb8a0102420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009900 tx=0x7fb89c02a410 comp rx=0 tx=0).stop 2024-04-25T01:48:04.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.359+0000 7fb8a84ba640 1 -- 172.21.15.78:0/398793670 shutdown_connections 2024-04-25T01:48:04.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.359+0000 7fb8a84ba640 1 --2- 172.21.15.78:0/398793670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a0105310 0x7fb8a0105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.359+0000 7fb8a84ba640 1 --2- 172.21.15.78:0/398793670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8a0102960 0x7fb8a0104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.359+0000 7fb8a84ba640 1 --2- 172.21.15.78:0/398793670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8a00ffff0 0x7fb8a0102420 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.359+0000 7fb8a84ba640 1 -- 172.21.15.78:0/398793670 >> 172.21.15.78:0/398793670 conn(0x7fb8a00fbbf0 msgr2=0x7fb8a00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:04.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.359+0000 7fb8a84ba640 1 -- 172.21.15.78:0/398793670 shutdown_connections 2024-04-25T01:48:04.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.359+0000 7fb8a84ba640 1 -- 172.21.15.78:0/398793670 wait complete. 2024-04-25T01:48:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a84ba640 1 Processor -- start 2024-04-25T01:48:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a84ba640 1 -- start start 2024-04-25T01:48:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a84ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a00ffff0 0x7fb8a019e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a84ba640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8a0102960 0x7fb8a019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a84ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8a0105310 0x7fb8a01a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a84ba640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01139d0 con 0x7fb8a0105310 2024-04-25T01:48:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a84ba640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0113850 con 0x7fb8a0102960 2024-04-25T01:48:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a84ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0113b50 con 0x7fb8a00ffff0 2024-04-25T01:48:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a6a57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8a0105310 0x7fb8a01a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a6a57640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8a0105310 0x7fb8a01a55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57088/0 (socket says 172.21.15.78:57088) 2024-04-25T01:48:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a6a57640 1 -- 172.21.15.78:0/3766867649 learned_addr learned my addr 172.21.15.78:0/3766867649 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:48:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a6256640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a00ffff0 0x7fb8a019e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a5a55640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8a0102960 0x7fb8a019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:04.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a6a57640 1 -- 172.21.15.78:0/3766867649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a00ffff0 msgr2=0x7fb8a019e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a6a57640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a00ffff0 0x7fb8a019e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a6a57640 1 -- 172.21.15.78:0/3766867649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8a0102960 msgr2=0x7fb8a019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a6a57640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8a0102960 0x7fb8a019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a6a57640 1 -- 172.21.15.78:0/3766867649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a01a5c50 con 0x7fb8a0105310 2024-04-25T01:48:04.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a5a55640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8a0102960 0x7fb8a019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:48:04.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.360+0000 7fb8a6a57640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8a0105310 0x7fb8a01a55e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb89800cce0 tx=0x7fb89800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:04.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.361+0000 7fb8977fe640 1 -- 172.21.15.78:0/3766867649 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89800f450 con 0x7fb8a0105310 2024-04-25T01:48:04.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.361+0000 7fb8977fe640 1 -- 172.21.15.78:0/3766867649 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb89800fad0 con 0x7fb8a0105310 2024-04-25T01:48:04.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.361+0000 7fb8a84ba640 1 -- 172.21.15.78:0/3766867649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a01a5f40 con 0x7fb8a0105310 2024-04-25T01:48:04.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.361+0000 7fb8977fe640 1 -- 172.21.15.78:0/3766867649 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898018c10 con 0x7fb8a0105310 2024-04-25T01:48:04.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.361+0000 7fb8a84ba640 1 -- 172.21.15.78:0/3766867649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a01a63b0 con 0x7fb8a0105310 2024-04-25T01:48:04.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.362+0000 7fb8a84ba640 1 -- 172.21.15.78:0/3766867649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a00fd6e0 con 0x7fb8a0105310 2024-04-25T01:48:04.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.363+0000 7fb8977fe640 1 -- 172.21.15.78:0/3766867649 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fb89800f5f0 con 0x7fb8a0105310 2024-04-25T01:48:04.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.363+0000 7fb8977fe640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb88c0a2b90 0x7fb88c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:04.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.363+0000 7fb8a6256640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb88c0a2b90 0x7fb88c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:04.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.363+0000 7fb8977fe640 1 -- 172.21.15.78:0/3766867649 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7fb8980cfc00 con 0x7fb8a0105310 2024-04-25T01:48:04.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.364+0000 7fb8a6256640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb88c0a2b90 0x7fb88c0a5050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb89c000c00 tx=0x7fb89c0315d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:04.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.367+0000 7fb8977fe640 1 -- 172.21.15.78:0/3766867649 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb898014030 con 0x7fb8a0105310 2024-04-25T01:48:04.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:04 smithi196 ceph-mon[32978]: pgmap v437: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T01:48:04.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:04 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3963585946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:48:04.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.540+0000 7fb8a84ba640 1 -- 172.21.15.78:0/3766867649 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb8a002d170 con 0x7fb8a0105310 2024-04-25T01:48:04.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.541+0000 7fb8977fe640 1 -- 172.21.15.78:0/3766867649 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fb89809c810 con 0x7fb8a0105310 2024-04-25T01:48:04.538 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:48:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.542+0000 7fb8a84ba640 1 -- 172.21.15.78:0/3766867649 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb88c0a2b90 msgr2=0x7fb88c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.542+0000 7fb8a84ba640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb88c0a2b90 0x7fb88c0a5050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb89c000c00 tx=0x7fb89c0315d0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.542+0000 7fb8a84ba640 1 -- 172.21.15.78:0/3766867649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8a0105310 msgr2=0x7fb8a01a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.542+0000 7fb8a84ba640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8a0105310 0x7fb8a01a55e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb89800cce0 tx=0x7fb89800a620 comp rx=0 tx=0).stop 2024-04-25T01:48:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.542+0000 7fb8a84ba640 1 -- 172.21.15.78:0/3766867649 shutdown_connections 2024-04-25T01:48:04.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.542+0000 7fb8a84ba640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fb88c0a2b90 0x7fb88c0a5050 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.542+0000 7fb8a84ba640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8a0105310 0x7fb8a01a55e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.542+0000 7fb8a84ba640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8a0102960 0x7fb8a019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.542+0000 7fb8a84ba640 1 --2- 172.21.15.78:0/3766867649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a00ffff0 0x7fb8a019e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.542+0000 7fb8a84ba640 1 -- 172.21.15.78:0/3766867649 >> 172.21.15.78:0/3766867649 conn(0x7fb8a00fbbf0 msgr2=0x7fb8a01032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.543+0000 7fb8a84ba640 1 -- 172.21.15.78:0/3766867649 shutdown_connections 2024-04-25T01:48:04.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.543+0000 7fb8a84ba640 1 -- 172.21.15.78:0/3766867649 wait complete. 2024-04-25T01:48:04.550 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:48:04.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.656+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/341704491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e58102230 msgr2=0x7f7e58108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.656+0000 7f7e5d3a2640 1 --2- 172.21.15.78:0/341704491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e58102230 0x7f7e58108ac0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009900 tx=0x7f7e5002a410 comp rx=0 tx=0).stop 2024-04-25T01:48:04.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.656+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/341704491 shutdown_connections 2024-04-25T01:48:04.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.656+0000 7f7e5d3a2640 1 --2- 172.21.15.78:0/341704491 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e58102230 0x7f7e58108ac0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.656+0000 7f7e5d3a2640 1 --2- 172.21.15.78:0/341704491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e58101870 0x7f7e58101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.656+0000 7f7e5d3a2640 1 --2- 172.21.15.78:0/341704491 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e58100680 0x7f7e58100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.656+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/341704491 >> 172.21.15.78:0/341704491 conn(0x7f7e580fbbf0 msgr2=0x7f7e580fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:04.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.656+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/341704491 shutdown_connections 2024-04-25T01:48:04.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.656+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/341704491 wait complete. 2024-04-25T01:48:04.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e5d3a2640 1 Processor -- start 2024-04-25T01:48:04.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e5d3a2640 1 -- start start 2024-04-25T01:48:04.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e5d3a2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e58100680 0x7f7e58069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:04.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e5d3a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e58101870 0x7f7e5819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:04.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e5d3a2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e58102230 0x7f7e5819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:04.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e5d3a2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5810c300 con 0x7f7e58101870 2024-04-25T01:48:04.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e5d3a2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5810c180 con 0x7f7e58102230 2024-04-25T01:48:04.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e5d3a2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5810c480 con 0x7f7e58100680 2024-04-25T01:48:04.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e567fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e58101870 0x7f7e5819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:04.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e567fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e58101870 0x7f7e5819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57108/0 (socket says 172.21.15.78:57108) 2024-04-25T01:48:04.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e567fc640 1 -- 172.21.15.78:0/3470047892 learned_addr learned my addr 172.21.15.78:0/3470047892 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:48:04.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e577fe640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e58102230 0x7f7e5819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:04.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e56ffd640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e58100680 0x7f7e58069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:04.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e567fc640 1 -- 172.21.15.78:0/3470047892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e58100680 msgr2=0x7f7e58069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e567fc640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e58100680 0x7f7e58069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e567fc640 1 -- 172.21.15.78:0/3470047892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e58102230 msgr2=0x7f7e5819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e567fc640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e58102230 0x7f7e5819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e567fc640 1 -- 172.21.15.78:0/3470047892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5819f6f0 con 0x7f7e58101870 2024-04-25T01:48:04.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.657+0000 7f7e56ffd640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e58100680 0x7f7e58069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:48:04.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.658+0000 7f7e567fc640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e58101870 0x7f7e5819c7c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00d950 tx=0x7f7e4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:04.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.658+0000 7f7e3ffff640 1 -- 172.21.15.78:0/3470047892 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c017070 con 0x7f7e58101870 2024-04-25T01:48:04.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.658+0000 7f7e3ffff640 1 -- 172.21.15.78:0/3470047892 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e4c009c30 con 0x7f7e58101870 2024-04-25T01:48:04.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.658+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/3470047892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5819f980 con 0x7f7e58101870 2024-04-25T01:48:04.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.658+0000 7f7e3ffff640 1 -- 172.21.15.78:0/3470047892 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c00f6b0 con 0x7f7e58101870 2024-04-25T01:48:04.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.658+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/3470047892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5819fed0 con 0x7f7e58101870 2024-04-25T01:48:04.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.659+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/3470047892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e58107a20 con 0x7f7e58101870 2024-04-25T01:48:04.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.660+0000 7f7e3ffff640 1 -- 172.21.15.78:0/3470047892 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f7e4c0109d0 con 0x7f7e58101870 2024-04-25T01:48:04.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.661+0000 7f7e3ffff640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7e400a2b90 0x7f7e400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:04.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.661+0000 7f7e56ffd640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7e400a2b90 0x7f7e400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:04.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.661+0000 7f7e3ffff640 1 -- 172.21.15.78:0/3470047892 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f7e4c0cf160 con 0x7f7e58101870 2024-04-25T01:48:04.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.661+0000 7f7e56ffd640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7e400a2b90 0x7f7e400a5050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7e48005fd0 tx=0x7f7e48008310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:04.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.665+0000 7f7e3ffff640 1 -- 172.21.15.78:0/3470047892 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7e4c009de0 con 0x7f7e58101870 2024-04-25T01:48:04.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.842+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/3470047892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7e58062340 con 0x7f7e58101870 2024-04-25T01:48:04.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.843+0000 7f7e3ffff640 1 -- 172.21.15.78:0/3470047892 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f7e4c09bce0 con 0x7f7e58101870 2024-04-25T01:48:04.941 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-04-25T01:48:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.844+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/3470047892 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7e400a2b90 msgr2=0x7f7e400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.844+0000 7f7e5d3a2640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7e400a2b90 0x7f7e400a5050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7e48005fd0 tx=0x7f7e48008310 comp rx=0 tx=0).stop 2024-04-25T01:48:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.845+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/3470047892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e58101870 msgr2=0x7f7e5819c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.845+0000 7f7e5d3a2640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e58101870 0x7f7e5819c7c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00d950 tx=0x7f7e4c00de10 comp rx=0 tx=0).stop 2024-04-25T01:48:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.845+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/3470047892 shutdown_connections 2024-04-25T01:48:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.845+0000 7f7e5d3a2640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e58102230 0x7f7e5819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.845+0000 7f7e5d3a2640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f7e400a2b90 0x7f7e400a5050 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.845+0000 7f7e5d3a2640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e58101870 0x7f7e5819c7c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.845+0000 7f7e5d3a2640 1 --2- 172.21.15.78:0/3470047892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e58100680 0x7f7e58069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.845+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/3470047892 >> 172.21.15.78:0/3470047892 conn(0x7f7e580fbbf0 msgr2=0x7f7e58104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:04.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.845+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/3470047892 shutdown_connections 2024-04-25T01:48:04.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:04.845+0000 7f7e5d3a2640 1 -- 172.21.15.78:0/3470047892 wait complete. 2024-04-25T01:48:04.944 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:48:05.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.054+0000 7f7704688640 1 -- 172.21.15.78:0/3360225275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc100680 msgr2=0x7f76fc100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:05.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.054+0000 7f7704688640 1 --2- 172.21.15.78:0/3360225275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc100680 0x7f76fc100aa0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009900 tx=0x7f76f802a410 comp rx=0 tx=0).stop 2024-04-25T01:48:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.054+0000 7f7704688640 1 -- 172.21.15.78:0/3360225275 shutdown_connections 2024-04-25T01:48:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.054+0000 7f7704688640 1 --2- 172.21.15.78:0/3360225275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76fc102230 0x7f76fc108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.054+0000 7f7704688640 1 --2- 172.21.15.78:0/3360225275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76fc101870 0x7f76fc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.054+0000 7f7704688640 1 --2- 172.21.15.78:0/3360225275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc100680 0x7f76fc100aa0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.054+0000 7f7704688640 1 -- 172.21.15.78:0/3360225275 >> 172.21.15.78:0/3360225275 conn(0x7f76fc0fbbf0 msgr2=0x7f76fc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.054+0000 7f7704688640 1 -- 172.21.15.78:0/3360225275 shutdown_connections 2024-04-25T01:48:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.054+0000 7f7704688640 1 -- 172.21.15.78:0/3360225275 wait complete. 2024-04-25T01:48:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7704688640 1 Processor -- start 2024-04-25T01:48:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7704688640 1 -- start start 2024-04-25T01:48:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7704688640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76fc100680 0x7f76fc196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:05.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7704688640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76fc101870 0x7f76fc196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:05.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7704688640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc102230 0x7f76fc19df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:05.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7704688640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc10c320 con 0x7f76fc102230 2024-04-25T01:48:05.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7704688640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc10c1a0 con 0x7f76fc101870 2024-04-25T01:48:05.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7704688640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc10c4a0 con 0x7f76fc100680 2024-04-25T01:48:05.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7702c25640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc102230 0x7f76fc19df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:05.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7702c25640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc102230 0x7f76fc19df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57114/0 (socket says 172.21.15.78:57114) 2024-04-25T01:48:05.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7702424640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76fc100680 0x7f76fc196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:05.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7702c25640 1 -- 172.21.15.78:0/2261425890 learned_addr learned my addr 172.21.15.78:0/2261425890 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:48:05.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7701c23640 1 --2- 172.21.15.78:0/2261425890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76fc101870 0x7f76fc196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:05.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7702c25640 1 -- 172.21.15.78:0/2261425890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76fc100680 msgr2=0x7f76fc196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:05.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7702c25640 1 --2- 172.21.15.78:0/2261425890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76fc100680 0x7f76fc196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7702c25640 1 -- 172.21.15.78:0/2261425890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76fc101870 msgr2=0x7f76fc196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:05.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7702c25640 1 --2- 172.21.15.78:0/2261425890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76fc101870 0x7f76fc196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7702c25640 1 -- 172.21.15.78:0/2261425890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76fc19e5b0 con 0x7f76fc102230 2024-04-25T01:48:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7701c23640 1 --2- 172.21.15.78:0/2261425890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76fc101870 0x7f76fc196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:48:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.055+0000 7f7702c25640 1 --2- 172.21.15.78:0/2261425890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc102230 0x7f76fc19df40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f76f4007cb0 tx=0x7f76f400e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.056+0000 7f76f37fe640 1 -- 172.21.15.78:0/2261425890 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f400eb30 con 0x7f76fc102230 2024-04-25T01:48:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.056+0000 7f76f37fe640 1 -- 172.21.15.78:0/2261425890 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f400ecd0 con 0x7f76fc102230 2024-04-25T01:48:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.056+0000 7f76f37fe640 1 -- 172.21.15.78:0/2261425890 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4015650 con 0x7f76fc102230 2024-04-25T01:48:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.056+0000 7f7704688640 1 -- 172.21.15.78:0/2261425890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76fc068930 con 0x7f76fc102230 2024-04-25T01:48:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.056+0000 7f7704688640 1 -- 172.21.15.78:0/2261425890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76fc068e30 con 0x7f76fc102230 2024-04-25T01:48:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.057+0000 7f7704688640 1 -- 172.21.15.78:0/2261425890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76fc068560 con 0x7f76fc102230 2024-04-25T01:48:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.058+0000 7f76f37fe640 1 -- 172.21.15.78:0/2261425890 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f76f400a440 con 0x7f76fc102230 2024-04-25T01:48:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.058+0000 7f76f37fe640 1 --2- 172.21.15.78:0/2261425890 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f76e80a2b90 0x7f76e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:05.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.058+0000 7f7702424640 1 --2- 172.21.15.78:0/2261425890 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f76e80a2b90 0x7f76e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.058+0000 7f76f37fe640 1 -- 172.21.15.78:0/2261425890 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f76f40d0fc0 con 0x7f76fc102230 2024-04-25T01:48:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.059+0000 7f7702424640 1 --2- 172.21.15.78:0/2261425890 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f76e80a2b90 0x7f76e80a5050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f76f802aa00 tx=0x7f76f80315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:05.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.063+0000 7f76f37fe640 1 -- 172.21.15.78:0/2261425890 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f76f409db40 con 0x7f76fc102230 2024-04-25T01:48:05.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.238+0000 7f7704688640 1 -- 172.21.15.78:0/2261425890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f76fc19e840 con 0x7f76fc102230 2024-04-25T01:48:05.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.239+0000 7f76f37fe640 1 -- 172.21.15.78:0/2261425890 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f76f409d440 con 0x7f76fc102230 2024-04-25T01:48:05.235 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-04-25T01:48:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.240+0000 7f7704688640 1 -- 172.21.15.78:0/2261425890 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f76e80a2b90 msgr2=0x7f76e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.240+0000 7f7704688640 1 --2- 172.21.15.78:0/2261425890 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f76e80a2b90 0x7f76e80a5050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f76f802aa00 tx=0x7f76f80315b0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.241+0000 7f7704688640 1 -- 172.21.15.78:0/2261425890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc102230 msgr2=0x7f76fc19df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.241+0000 7f7704688640 1 --2- 172.21.15.78:0/2261425890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc102230 0x7f76fc19df40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f76f4007cb0 tx=0x7f76f400e420 comp rx=0 tx=0).stop 2024-04-25T01:48:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.241+0000 7f7704688640 1 -- 172.21.15.78:0/2261425890 shutdown_connections 2024-04-25T01:48:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.241+0000 7f7704688640 1 --2- 172.21.15.78:0/2261425890 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f76e80a2b90 0x7f76e80a5050 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.241+0000 7f7704688640 1 --2- 172.21.15.78:0/2261425890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76fc102230 0x7f76fc19df40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.241+0000 7f7704688640 1 --2- 172.21.15.78:0/2261425890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76fc101870 0x7f76fc196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.241+0000 7f7704688640 1 --2- 172.21.15.78:0/2261425890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76fc100680 0x7f76fc196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.241+0000 7f7704688640 1 -- 172.21.15.78:0/2261425890 >> 172.21.15.78:0/2261425890 conn(0x7f76fc0fbbf0 msgr2=0x7f76fc104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.241+0000 7f7704688640 1 -- 172.21.15.78:0/2261425890 shutdown_connections 2024-04-25T01:48:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.241+0000 7f7704688640 1 -- 172.21.15.78:0/2261425890 wait complete. 2024-04-25T01:48:05.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:05 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/773413744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:48:05.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:05 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3766867649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:48:05.248 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:05 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3470047892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:48:05.249 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:48:05.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:05 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/773413744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:48:05.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:05 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3766867649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:48:05.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:05 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3470047892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:48:05.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 -- 172.21.15.78:0/2393196342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95f4102250 msgr2=0x7f95f4108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:05.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 --2- 172.21.15.78:0/2393196342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95f4102250 0x7f95f4108ae0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009ca0 tx=0x7f95ec02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 -- 172.21.15.78:0/2393196342 shutdown_connections 2024-04-25T01:48:05.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 --2- 172.21.15.78:0/2393196342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95f4102250 0x7f95f4108ae0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 --2- 172.21.15.78:0/2393196342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95f4101890 0x7f95f4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 --2- 172.21.15.78:0/2393196342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95f41006a0 0x7f95f4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 -- 172.21.15.78:0/2393196342 >> 172.21.15.78:0/2393196342 conn(0x7f95f40fbc30 msgr2=0x7f95f40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:05.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 -- 172.21.15.78:0/2393196342 shutdown_connections 2024-04-25T01:48:05.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 -- 172.21.15.78:0/2393196342 wait complete. 2024-04-25T01:48:05.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 Processor -- start 2024-04-25T01:48:05.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 -- start start 2024-04-25T01:48:05.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95f41006a0 0x7f95f4199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:05.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95f4101890 0x7f95f419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:05.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95f4102250 0x7f95f41a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:05.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95f410c320 con 0x7f95f4102250 2024-04-25T01:48:05.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f95f410c1a0 con 0x7f95f41006a0 2024-04-25T01:48:05.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.352+0000 7f95fb49a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95f410c4a0 con 0x7f95f4101890 2024-04-25T01:48:05.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95f9a37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95f4102250 0x7f95f41a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:05.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95f9a37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95f4102250 0x7f95f41a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57132/0 (socket says 172.21.15.78:57132) 2024-04-25T01:48:05.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95f9a37640 1 -- 172.21.15.78:0/3209221953 learned_addr learned my addr 172.21.15.78:0/3209221953 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:48:05.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95f8a35640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95f4101890 0x7f95f419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:05.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95f9236640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95f41006a0 0x7f95f4199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:05.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95f9a37640 1 -- 172.21.15.78:0/3209221953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95f4101890 msgr2=0x7f95f419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:05.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95f9a37640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95f4101890 0x7f95f419a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95f9a37640 1 -- 172.21.15.78:0/3209221953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95f41006a0 msgr2=0x7f95f4199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:05.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95f9a37640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95f41006a0 0x7f95f4199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95f9a37640 1 -- 172.21.15.78:0/3209221953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f41a1850 con 0x7f95f4102250 2024-04-25T01:48:05.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95f9236640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95f41006a0 0x7f95f4199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:48:05.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95f9a37640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95f4102250 0x7f95f41a11b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f95ec005e30 tx=0x7f95ec031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:05.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95e67fc640 1 -- 172.21.15.78:0/3209221953 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec003c30 con 0x7f95f4102250 2024-04-25T01:48:05.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95fb49a640 1 -- 172.21.15.78:0/3209221953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f41a1ae0 con 0x7f95f4102250 2024-04-25T01:48:05.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95e67fc640 1 -- 172.21.15.78:0/3209221953 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ec003dd0 con 0x7f95f4102250 2024-04-25T01:48:05.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95e67fc640 1 -- 172.21.15.78:0/3209221953 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec02fde0 con 0x7f95f4102250 2024-04-25T01:48:05.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.353+0000 7f95fb49a640 1 -- 172.21.15.78:0/3209221953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f41a2080 con 0x7f95f4102250 2024-04-25T01:48:05.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.354+0000 7f95fb49a640 1 -- 172.21.15.78:0/3209221953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f41a2250 con 0x7f95f4102250 2024-04-25T01:48:05.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.355+0000 7f95e67fc640 1 -- 172.21.15.78:0/3209221953 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f95ec004050 con 0x7f95f4102250 2024-04-25T01:48:05.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.356+0000 7f95e67fc640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f95e00a2b90 0x7f95e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:05.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.356+0000 7f95f9236640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f95e00a2b90 0x7f95e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:05.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.356+0000 7f95e67fc640 1 -- 172.21.15.78:0/3209221953 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f95ec0f08d0 con 0x7f95f4102250 2024-04-25T01:48:05.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.356+0000 7f95f9236640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f95e00a2b90 0x7f95e00a5050 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f95f419b110 tx=0x7f95f0005dd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.360+0000 7f95e67fc640 1 -- 172.21.15.78:0/3209221953 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f95ec0bd450 con 0x7f95f4102250 2024-04-25T01:48:05.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:05 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/773413744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:48:05.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:05 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3766867649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T01:48:05.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:05 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3470047892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T01:48:05.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.554+0000 7f95fb49a640 1 -- 172.21.15.78:0/3209221953 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f95f41a2420 con 0x7f95f4102250 2024-04-25T01:48:05.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.554+0000 7f95e67fc640 1 -- 172.21.15.78:0/3209221953 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+589 (secure 0 0 0) 0x7f95ec0bcd50 con 0x7f95f4102250 2024-04-25T01:48:05.551 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1101495738 2024-04-26T01:36:16.812502+0000 2024-04-25T01:48:05.551 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1101495738 2024-04-26T01:36:16.812502+0000 2024-04-25T01:48:05.551 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2762765405 2024-04-26T01:36:16.812502+0000 2024-04-25T01:48:05.551 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/4170716207 2024-04-26T01:35:50.507447+0000 2024-04-25T01:48:05.551 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1136116795 2024-04-26T01:35:50.507447+0000 2024-04-25T01:48:05.551 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1544860340 2024-04-26T01:35:50.507447+0000 2024-04-25T01:48:05.551 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1544860340 2024-04-26T01:35:50.507447+0000 2024-04-25T01:48:05.551 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/211357528 2024-04-26T01:36:16.812502+0000 2024-04-25T01:48:05.551 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/88994848 2024-04-26T01:36:16.812502+0000 2024-04-25T01:48:05.551 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1196975031 2024-04-26T01:35:50.507447+0000 2024-04-25T01:48:05.551 INFO:teuthology.orchestra.run.smithi078.stderr:listed 10 entries 2024-04-25T01:48:05.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.556+0000 7f95fb49a640 1 -- 172.21.15.78:0/3209221953 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f95e00a2b90 msgr2=0x7f95e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:05.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.556+0000 7f95fb49a640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f95e00a2b90 0x7f95e00a5050 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f95f419b110 tx=0x7f95f0005dd0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.556+0000 7f95fb49a640 1 -- 172.21.15.78:0/3209221953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95f4102250 msgr2=0x7f95f41a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:05.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.556+0000 7f95fb49a640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95f4102250 0x7f95f41a11b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f95ec005e30 tx=0x7f95ec031ee0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.556+0000 7f95fb49a640 1 -- 172.21.15.78:0/3209221953 shutdown_connections 2024-04-25T01:48:05.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.556+0000 7f95fb49a640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f95e00a2b90 0x7f95e00a5050 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.556+0000 7f95fb49a640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95f4102250 0x7f95f41a11b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.556+0000 7f95fb49a640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95f4101890 0x7f95f419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.556+0000 7f95fb49a640 1 --2- 172.21.15.78:0/3209221953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95f41006a0 0x7f95f4199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.556+0000 7f95fb49a640 1 -- 172.21.15.78:0/3209221953 >> 172.21.15.78:0/3209221953 conn(0x7f95f40fbc30 msgr2=0x7f95f40fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:05.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.556+0000 7f95fb49a640 1 -- 172.21.15.78:0/3209221953 shutdown_connections 2024-04-25T01:48:05.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.556+0000 7f95fb49a640 1 -- 172.21.15.78:0/3209221953 wait complete. 2024-04-25T01:48:05.564 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T01:48:05.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:48:05.564 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T01:48:05.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T01:48:05.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:48:05.604 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T01:48:05.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:48:05.659 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-04-25T01:48:05.714 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T01:48:05.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:48:05.715 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:48:05.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:48:05.778 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-04-25T01:48:05.876 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T01:48:05.888 INFO:tasks.cephadm:Teardown begin 2024-04-25T01:48:05.888 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T01:48:05.888 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:48:05.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.991+0000 7f8592a45640 1 -- 172.21.15.78:0/4249241788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c102980 msgr2=0x7f858c102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:05.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.991+0000 7f8592a45640 1 --2- 172.21.15.78:0/4249241788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c102980 0x7f858c102e00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8580009900 tx=0x7f858002a410 comp rx=0 tx=0).stop 2024-04-25T01:48:05.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.991+0000 7f8592a45640 1 -- 172.21.15.78:0/4249241788 shutdown_connections 2024-04-25T01:48:05.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.991+0000 7f8592a45640 1 --2- 172.21.15.78:0/4249241788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f858c103340 0x7f858c109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.991+0000 7f8592a45640 1 --2- 172.21.15.78:0/4249241788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c102980 0x7f858c102e00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.991+0000 7f8592a45640 1 --2- 172.21.15.78:0/4249241788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f858c101790 0x7f858c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.991+0000 7f8592a45640 1 -- 172.21.15.78:0/4249241788 >> 172.21.15.78:0/4249241788 conn(0x7f858c0fcd40 msgr2=0x7f858c0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:05.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.991+0000 7f8592a45640 1 -- 172.21.15.78:0/4249241788 shutdown_connections 2024-04-25T01:48:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.991+0000 7f8592a45640 1 -- 172.21.15.78:0/4249241788 wait complete. 2024-04-25T01:48:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.992+0000 7f8592a45640 1 Processor -- start 2024-04-25T01:48:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.992+0000 7f8592a45640 1 -- start start 2024-04-25T01:48:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.992+0000 7f8592a45640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f858c101790 0x7f858c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.992+0000 7f8592a45640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c102980 0x7f858c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.992+0000 7f8592a45640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f858c103340 0x7f858c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.992+0000 7f8592a45640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f858c074e20 con 0x7f858c102980 2024-04-25T01:48:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.992+0000 7f8592a45640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f858c074ca0 con 0x7f858c101790 2024-04-25T01:48:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.992+0000 7f8592a45640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f858c074fa0 con 0x7f858c103340 2024-04-25T01:48:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.992+0000 7f858b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c102980 0x7f858c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:05.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.992+0000 7f858b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c102980 0x7f858c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57150/0 (socket says 172.21.15.78:57150) 2024-04-25T01:48:05.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.992+0000 7f858b7fe640 1 -- 172.21.15.78:0/4283543837 learned_addr learned my addr 172.21.15.78:0/4283543837 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:48:05.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.992+0000 7f858bfff640 1 --2- 172.21.15.78:0/4283543837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f858c101790 0x7f858c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:05.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.992+0000 7f8590fe2640 1 --2- 172.21.15.78:0/4283543837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f858c103340 0x7f858c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:05.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.993+0000 7f858b7fe640 1 -- 172.21.15.78:0/4283543837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f858c103340 msgr2=0x7f858c1a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:05.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.993+0000 7f858b7fe640 1 --2- 172.21.15.78:0/4283543837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f858c103340 0x7f858c1a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.993+0000 7f858b7fe640 1 -- 172.21.15.78:0/4283543837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f858c101790 msgr2=0x7f858c199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:05.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.993+0000 7f858b7fe640 1 --2- 172.21.15.78:0/4283543837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f858c101790 0x7f858c199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:05.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.993+0000 7f858b7fe640 1 -- 172.21.15.78:0/4283543837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858c1a1900 con 0x7f858c102980 2024-04-25T01:48:05.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.993+0000 7f858b7fe640 1 --2- 172.21.15.78:0/4283543837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c102980 0x7f858c19a230 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8580009230 tx=0x7f8580004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:05.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.993+0000 7f85897fa640 1 -- 172.21.15.78:0/4283543837 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858003f030 con 0x7f858c102980 2024-04-25T01:48:05.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.993+0000 7f85897fa640 1 -- 172.21.15.78:0/4283543837 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8580003ce0 con 0x7f858c102980 2024-04-25T01:48:05.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.993+0000 7f85897fa640 1 -- 172.21.15.78:0/4283543837 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8580035070 con 0x7f858c102980 2024-04-25T01:48:05.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.993+0000 7f8592a45640 1 -- 172.21.15.78:0/4283543837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858c1a1b90 con 0x7f858c102980 2024-04-25T01:48:05.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.993+0000 7f8592a45640 1 -- 172.21.15.78:0/4283543837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f858c1a20d0 con 0x7f858c102980 2024-04-25T01:48:05.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.994+0000 7f8592a45640 1 -- 172.21.15.78:0/4283543837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f858c108610 con 0x7f858c102980 2024-04-25T01:48:05.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.995+0000 7f85897fa640 1 -- 172.21.15.78:0/4283543837 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f8580042030 con 0x7f858c102980 2024-04-25T01:48:05.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.996+0000 7f85897fa640 1 --2- 172.21.15.78:0/4283543837 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f85740a2c10 0x7f85740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:05.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.996+0000 7f858bfff640 1 --2- 172.21.15.78:0/4283543837 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f85740a2c10 0x7f85740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:05.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.996+0000 7f85897fa640 1 -- 172.21.15.78:0/4283543837 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f85800ef670 con 0x7f858c102980 2024-04-25T01:48:05.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:05.996+0000 7f858bfff640 1 --2- 172.21.15.78:0/4283543837 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f85740a2c10 0x7f85740a50d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f858c19b1c0 tx=0x7f857c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:05.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.000+0000 7f85897fa640 1 -- 172.21.15.78:0/4283543837 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f85800bbba0 con 0x7f858c102980 2024-04-25T01:48:06.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.226+0000 7f8592a45640 1 -- 172.21.15.78:0/4283543837 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f858c0623e0 con 0x7f858c102980 2024-04-25T01:48:06.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.227+0000 7f85897fa640 1 -- 172.21.15.78:0/4283543837 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11944 (secure 0 0 0) 0x7f85800bfea0 con 0x7f858c102980 2024-04-25T01:48:06.224 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:48:06.224 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:43:17.355192+0000","modified":"2024-04-25T01:45:38.363323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24455,"mds_2":24448,"mds_3":24392,"mds_4":24379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1337519666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1337519666},{"type":"v1","addr":"172.21.15.78:6839","nonce":1337519666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1963410683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1963410683},{"type":"v1","addr":"172.21.15.78:6841","nonce":1963410683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24379":{"gid":24379,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1969344843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1969344843},{"type":"v1","addr":"172.21.15.152:6839","nonce":1969344843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24448,24392,24455,24379,24491]},"id":1}]} 2024-04-25T01:48:06.224 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-04-25T01:48:06.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.229+0000 7f8592a45640 1 -- 172.21.15.78:0/4283543837 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f85740a2c10 msgr2=0x7f85740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.229+0000 7f8592a45640 1 --2- 172.21.15.78:0/4283543837 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f85740a2c10 0x7f85740a50d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f858c19b1c0 tx=0x7f857c00b390 comp rx=0 tx=0).stop 2024-04-25T01:48:06.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.229+0000 7f8592a45640 1 -- 172.21.15.78:0/4283543837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c102980 msgr2=0x7f858c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.229+0000 7f8592a45640 1 --2- 172.21.15.78:0/4283543837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c102980 0x7f858c19a230 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8580009230 tx=0x7f8580004360 comp rx=0 tx=0).stop 2024-04-25T01:48:06.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.229+0000 7f8592a45640 1 -- 172.21.15.78:0/4283543837 shutdown_connections 2024-04-25T01:48:06.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.229+0000 7f8592a45640 1 --2- 172.21.15.78:0/4283543837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f858c103340 0x7f858c1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.229+0000 7f8592a45640 1 --2- 172.21.15.78:0/4283543837 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f85740a2c10 0x7f85740a50d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.229+0000 7f8592a45640 1 --2- 172.21.15.78:0/4283543837 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c102980 0x7f858c19a230 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.229+0000 7f8592a45640 1 --2- 172.21.15.78:0/4283543837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f858c101790 0x7f858c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.229+0000 7f8592a45640 1 -- 172.21.15.78:0/4283543837 >> 172.21.15.78:0/4283543837 conn(0x7f858c0fcd40 msgr2=0x7f858c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:06.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.229+0000 7f8592a45640 1 -- 172.21.15.78:0/4283543837 shutdown_connections 2024-04-25T01:48:06.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.229+0000 7f8592a45640 1 -- 172.21.15.78:0/4283543837 wait complete. 2024-04-25T01:48:06.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:06 smithi078 ceph-mon[29469]: pgmap v438: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:48:06.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:06 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2261425890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:48:06.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:06 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3209221953' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T01:48:06.236 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:48:06.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:06 smithi152 ceph-mon[32465]: pgmap v438: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:48:06.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:06 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2261425890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:48:06.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:06 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3209221953' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T01:48:06.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.337+0000 7f37fc607640 1 -- 172.21.15.78:0/1842201272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f4101880 msgr2=0x7f37f4101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.337+0000 7f37fc607640 1 --2- 172.21.15.78:0/1842201272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f4101880 0x7f37f4101d00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009900 tx=0x7f37f002a410 comp rx=0 tx=0).stop 2024-04-25T01:48:06.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 -- 172.21.15.78:0/1842201272 shutdown_connections 2024-04-25T01:48:06.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 --2- 172.21.15.78:0/1842201272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37f4102240 0x7f37f4108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 --2- 172.21.15.78:0/1842201272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f4101880 0x7f37f4101d00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 --2- 172.21.15.78:0/1842201272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f4100690 0x7f37f4100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 -- 172.21.15.78:0/1842201272 >> 172.21.15.78:0/1842201272 conn(0x7f37f40fbc20 msgr2=0x7f37f40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:06.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 -- 172.21.15.78:0/1842201272 shutdown_connections 2024-04-25T01:48:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 -- 172.21.15.78:0/1842201272 wait complete. 2024-04-25T01:48:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 Processor -- start 2024-04-25T01:48:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 -- start start 2024-04-25T01:48:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f4100690 0x7f37f4197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f4101880 0x7f37f4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37f4102240 0x7f37f419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37f410d430 con 0x7f37f4101880 2024-04-25T01:48:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f37f410d2b0 con 0x7f37f4102240 2024-04-25T01:48:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37fc607640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37f410d5b0 con 0x7f37f4100690 2024-04-25T01:48:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37f9ba2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f4101880 0x7f37f4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37f9ba2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f4101880 0x7f37f4197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57176/0 (socket says 172.21.15.78:57176) 2024-04-25T01:48:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.338+0000 7f37f9ba2640 1 -- 172.21.15.78:0/737832893 learned_addr learned my addr 172.21.15.78:0/737832893 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:48:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.339+0000 7f37faba4640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37f4102240 0x7f37f419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.339+0000 7f37fa3a3640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f4100690 0x7f37f4197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:06.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.339+0000 7f37f9ba2640 1 -- 172.21.15.78:0/737832893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f4100690 msgr2=0x7f37f4197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.339+0000 7f37f9ba2640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f4100690 0x7f37f4197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.339+0000 7f37f9ba2640 1 -- 172.21.15.78:0/737832893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37f4102240 msgr2=0x7f37f419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.339+0000 7f37f9ba2640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37f4102240 0x7f37f419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.339+0000 7f37f9ba2640 1 -- 172.21.15.78:0/737832893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f419f680 con 0x7f37f4101880 2024-04-25T01:48:06.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.339+0000 7f37fa3a3640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f4100690 0x7f37f4197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:48:06.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.339+0000 7f37f9ba2640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f4101880 0x7f37f4197fa0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009230 tx=0x7f37f0004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:06.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.339+0000 7f37eb7fe640 1 -- 172.21.15.78:0/737832893 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f003f030 con 0x7f37f4101880 2024-04-25T01:48:06.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.339+0000 7f37eb7fe640 1 -- 172.21.15.78:0/737832893 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f0003ce0 con 0x7f37f4101880 2024-04-25T01:48:06.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.339+0000 7f37fc607640 1 -- 172.21.15.78:0/737832893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f419f910 con 0x7f37f4101880 2024-04-25T01:48:06.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.339+0000 7f37eb7fe640 1 -- 172.21.15.78:0/737832893 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f0035070 con 0x7f37f4101880 2024-04-25T01:48:06.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.339+0000 7f37fc607640 1 -- 172.21.15.78:0/737832893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f419fe00 con 0x7f37f4101880 2024-04-25T01:48:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.340+0000 7f37fc607640 1 -- 172.21.15.78:0/737832893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f41079b0 con 0x7f37f4101880 2024-04-25T01:48:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.341+0000 7f37eb7fe640 1 -- 172.21.15.78:0/737832893 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f37f0042030 con 0x7f37f4101880 2024-04-25T01:48:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.342+0000 7f37eb7fe640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f37e00a2b90 0x7f37e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.342+0000 7f37fa3a3640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f37e00a2b90 0x7f37e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.342+0000 7f37eb7fe640 1 -- 172.21.15.78:0/737832893 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f37f00efec0 con 0x7f37f4101880 2024-04-25T01:48:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.342+0000 7f37fa3a3640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f37e00a2b90 0x7f37e00a5050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f37e40058c0 tx=0x7f37e4005d50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:06.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.346+0000 7f37eb7fe640 1 -- 172.21.15.78:0/737832893 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f37f00bca40 con 0x7f37f4101880 2024-04-25T01:48:06.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:06 smithi196 ceph-mon[32978]: pgmap v438: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-25T01:48:06.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:06 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2261425890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T01:48:06.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:06 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3209221953' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T01:48:06.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.561+0000 7f37fc607640 1 -- 172.21.15.78:0/737832893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37f4062340 con 0x7f37f4101880 2024-04-25T01:48:06.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.562+0000 7f37eb7fe640 1 -- 172.21.15.78:0/737832893 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11944 (secure 0 0 0) 0x7f37f00bc340 con 0x7f37f4101880 2024-04-25T01:48:06.559 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:48:06.559 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2013480406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2013480406},{"type":"v1","addr":"172.21.15.78:6835","nonce":2013480406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3364816743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3364816743},{"type":"v1","addr":"172.21.15.78:6837","nonce":3364816743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:43:17.355192+0000","modified":"2024-04-25T01:45:38.363323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24491,"mds_1":24455,"mds_2":24448,"mds_3":24392,"mds_4":24379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1337519666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1337519666},{"type":"v1","addr":"172.21.15.78:6839","nonce":1337519666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1963410683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1963410683},{"type":"v1","addr":"172.21.15.78:6841","nonce":1963410683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24379":{"gid":24379,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.196:6833/4110091360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":4110091360},{"type":"v1","addr":"172.21.15.196:6833","nonce":4110091360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.152:6833/923579368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":923579368},{"type":"v1","addr":"172.21.15.152:6833","nonce":923579368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/2927651724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2927651724},{"type":"v1","addr":"172.21.15.196:6835","nonce":2927651724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/4251210610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":4251210610},{"type":"v1","addr":"172.21.15.152:6835","nonce":4251210610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6837/1117133390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":1117133390},{"type":"v1","addr":"172.21.15.196:6837","nonce":1117133390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.196:6839/460829167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":460829167},{"type":"v1","addr":"172.21.15.196:6839","nonce":460829167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6837/4112159743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":4112159743},{"type":"v1","addr":"172.21.15.152:6837","nonce":4112159743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6839/1969344843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":1969344843},{"type":"v1","addr":"172.21.15.152:6839","nonce":1969344843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24448,24392,24455,24379,24491]},"id":1}]} 2024-04-25T01:48:06.559 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-04-25T01:48:06.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.564+0000 7f37fc607640 1 -- 172.21.15.78:0/737832893 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f37e00a2b90 msgr2=0x7f37e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.564+0000 7f37fc607640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f37e00a2b90 0x7f37e00a5050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f37e40058c0 tx=0x7f37e4005d50 comp rx=0 tx=0).stop 2024-04-25T01:48:06.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.565+0000 7f37fc607640 1 -- 172.21.15.78:0/737832893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f4101880 msgr2=0x7f37f4197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.565+0000 7f37fc607640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f4101880 0x7f37f4197fa0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009230 tx=0x7f37f0004360 comp rx=0 tx=0).stop 2024-04-25T01:48:06.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.565+0000 7f37fc607640 1 -- 172.21.15.78:0/737832893 shutdown_connections 2024-04-25T01:48:06.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.565+0000 7f37fc607640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37f4102240 0x7f37f419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.565+0000 7f37fc607640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f37e00a2b90 0x7f37e00a5050 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.565+0000 7f37fc607640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37f4101880 0x7f37f4197fa0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.565+0000 7f37fc607640 1 --2- 172.21.15.78:0/737832893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37f4100690 0x7f37f4197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.565+0000 7f37fc607640 1 -- 172.21.15.78:0/737832893 >> 172.21.15.78:0/737832893 conn(0x7f37f40fbc20 msgr2=0x7f37f4104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:06.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.565+0000 7f37fc607640 1 -- 172.21.15.78:0/737832893 shutdown_connections 2024-04-25T01:48:06.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.565+0000 7f37fc607640 1 -- 172.21.15.78:0/737832893 wait complete. 2024-04-25T01:48:06.572 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:48:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.674+0000 7f4cc51af640 1 -- 172.21.15.78:0/1053928254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0103350 msgr2=0x7f4cc0109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.674+0000 7f4cc51af640 1 --2- 172.21.15.78:0/1053928254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0103350 0x7f4cc0109be0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0009900 tx=0x7f4cb002a410 comp rx=0 tx=0).stop 2024-04-25T01:48:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.674+0000 7f4cc51af640 1 -- 172.21.15.78:0/1053928254 shutdown_connections 2024-04-25T01:48:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.674+0000 7f4cc51af640 1 --2- 172.21.15.78:0/1053928254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0103350 0x7f4cc0109be0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.674+0000 7f4cc51af640 1 --2- 172.21.15.78:0/1053928254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cc0102990 0x7f4cc0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.674+0000 7f4cc51af640 1 --2- 172.21.15.78:0/1053928254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc01017a0 0x7f4cc0101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.674+0000 7f4cc51af640 1 -- 172.21.15.78:0/1053928254 >> 172.21.15.78:0/1053928254 conn(0x7f4cc00fccf0 msgr2=0x7f4cc00ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:06.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.674+0000 7f4cc51af640 1 -- 172.21.15.78:0/1053928254 shutdown_connections 2024-04-25T01:48:06.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.674+0000 7f4cc51af640 1 -- 172.21.15.78:0/1053928254 wait complete. 2024-04-25T01:48:06.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.675+0000 7f4cc51af640 1 Processor -- start 2024-04-25T01:48:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.675+0000 7f4cc51af640 1 -- start start 2024-04-25T01:48:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.675+0000 7f4cc51af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc01017a0 0x7f4cc0197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.675+0000 7f4cc51af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0102990 0x7f4cc0197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.675+0000 7f4cc51af640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cc0103350 0x7f4cc019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.675+0000 7f4cc51af640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc010d400 con 0x7f4cc0102990 2024-04-25T01:48:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.675+0000 7f4cc51af640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc010d280 con 0x7f4cc0103350 2024-04-25T01:48:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.675+0000 7f4cc51af640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc010d580 con 0x7f4cc01017a0 2024-04-25T01:48:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.675+0000 7f4cbe59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0102990 0x7f4cc0197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.675+0000 7f4cbe59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0102990 0x7f4cc0197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57204/0 (socket says 172.21.15.78:57204) 2024-04-25T01:48:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.675+0000 7f4cbe59c640 1 -- 172.21.15.78:0/3055672038 learned_addr learned my addr 172.21.15.78:0/3055672038 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:48:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.676+0000 7f4cbf59e640 1 --2- 172.21.15.78:0/3055672038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cc0103350 0x7f4cc019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.676+0000 7f4cbed9d640 1 --2- 172.21.15.78:0/3055672038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc01017a0 0x7f4cc0197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:06.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.676+0000 7f4cbe59c640 1 -- 172.21.15.78:0/3055672038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc01017a0 msgr2=0x7f4cc0197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.676+0000 7f4cbe59c640 1 --2- 172.21.15.78:0/3055672038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc01017a0 0x7f4cc0197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.676+0000 7f4cbe59c640 1 -- 172.21.15.78:0/3055672038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cc0103350 msgr2=0x7f4cc019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.676+0000 7f4cbe59c640 1 --2- 172.21.15.78:0/3055672038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cc0103350 0x7f4cc019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.676+0000 7f4cbe59c640 1 -- 172.21.15.78:0/3055672038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc019f660 con 0x7f4cc0102990 2024-04-25T01:48:06.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.676+0000 7f4cbe59c640 1 --2- 172.21.15.78:0/3055672038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0102990 0x7f4cc0197f80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400ebc0 tx=0x7f4cb400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:06.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.676+0000 7f4ca7fff640 1 -- 172.21.15.78:0/3055672038 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4010630 con 0x7f4cc0102990 2024-04-25T01:48:06.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.676+0000 7f4ca7fff640 1 -- 172.21.15.78:0/3055672038 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb4010cb0 con 0x7f4cc0102990 2024-04-25T01:48:06.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.676+0000 7f4cc51af640 1 -- 172.21.15.78:0/3055672038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc019f950 con 0x7f4cc0102990 2024-04-25T01:48:06.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.676+0000 7f4ca7fff640 1 -- 172.21.15.78:0/3055672038 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4018bd0 con 0x7f4cc0102990 2024-04-25T01:48:06.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.676+0000 7f4cc51af640 1 -- 172.21.15.78:0/3055672038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc019fea0 con 0x7f4cc0102990 2024-04-25T01:48:06.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.677+0000 7f4cc51af640 1 -- 172.21.15.78:0/3055672038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc0108a60 con 0x7f4cc0102990 2024-04-25T01:48:06.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.678+0000 7f4ca7fff640 1 -- 172.21.15.78:0/3055672038 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4cb40107d0 con 0x7f4cc0102990 2024-04-25T01:48:06.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.679+0000 7f4ca7fff640 1 --2- 172.21.15.78:0/3055672038 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4ca80a2b90 0x7f4ca80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:06.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.679+0000 7f4cbed9d640 1 --2- 172.21.15.78:0/3055672038 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4ca80a2b90 0x7f4ca80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:06.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.679+0000 7f4ca7fff640 1 -- 172.21.15.78:0/3055672038 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7f4cb40cfa40 con 0x7f4cc0102990 2024-04-25T01:48:06.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.679+0000 7f4cbed9d640 1 --2- 172.21.15.78:0/3055672038 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4ca80a2b90 0x7f4ca80a5050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4cc01027f0 tx=0x7f4cb800a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:06.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.684+0000 7f4ca7fff640 1 -- 172.21.15.78:0/3055672038 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4cb409bf70 con 0x7f4cc0102990 2024-04-25T01:48:06.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.861+0000 7f4cc51af640 1 -- 172.21.15.78:0/3055672038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cc0062340 con 0x7f4cc0102990 2024-04-25T01:48:06.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.863+0000 7f4ca7fff640 1 -- 172.21.15.78:0/3055672038 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24844 (secure 0 0 0) 0x7f4cb40a0270 con 0x7f4cc0102990 2024-04-25T01:48:06.859 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-25T01:48:06.860 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":94,"fsid":"df36bea8-02a3-11ef-bc93-c7b262605968","created":"2024-04-25T01:35:22.529144+0000","modified":"2024-04-25T01:45:18.989934+0000","last_up_change":"2024-04-25T01:42:27.422399+0000","last_in_change":"2024-04-25T01:42:07.316077+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:38:39.136418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:43:15.613881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:43:16.600270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:43:19.650457+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bab2e70b-bf93-48c8-944f-00e2d8424353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6803","nonce":519023653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6805","nonce":519023653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6809","nonce":519023653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":519023653},{"type":"v1","addr":"172.21.15.78:6807","nonce":519023653}]},"public_addr":"172.21.15.78:6803/519023653","cluster_addr":"172.21.15.78:6805/519023653","heartbeat_back_addr":"172.21.15.78:6809/519023653","heartbeat_front_addr":"172.21.15.78:6807/519023653","state":["exists","up"]},{"osd":1,"uuid":"54418993-9619-4854-9ce9-4bc5ff572a10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6801","nonce":2731006494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6803","nonce":2731006494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6807","nonce":2731006494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":2731006494},{"type":"v1","addr":"172.21.15.152:6805","nonce":2731006494}]},"public_addr":"172.21.15.152:6801/2731006494","cluster_addr":"172.21.15.152:6803/2731006494","heartbeat_back_addr":"172.21.15.152:6807/2731006494","heartbeat_front_addr":"172.21.15.152:6805/2731006494","state":["exists","up"]},{"osd":2,"uuid":"fcc6ae83-6a79-4c84-bf51-d4c6a1c15630","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6801","nonce":2032225723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6803","nonce":2032225723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6807","nonce":2032225723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2032225723},{"type":"v1","addr":"172.21.15.196:6805","nonce":2032225723}]},"public_addr":"172.21.15.196:6801/2032225723","cluster_addr":"172.21.15.196:6803/2032225723","heartbeat_back_addr":"172.21.15.196:6807/2032225723","heartbeat_front_addr":"172.21.15.196:6805/2032225723","state":["exists","up"]},{"osd":3,"uuid":"065344a4-91bb-4940-821e-1d7abd0dc292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6811","nonce":3760935265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6813","nonce":3760935265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6817","nonce":3760935265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":3760935265},{"type":"v1","addr":"172.21.15.78:6815","nonce":3760935265}]},"public_addr":"172.21.15.78:6811/3760935265","cluster_addr":"172.21.15.78:6813/3760935265","heartbeat_back_addr":"172.21.15.78:6817/3760935265","heartbeat_front_addr":"172.21.15.78:6815/3760935265","state":["exists","up"]},{"osd":4,"uuid":"6def52d6-f070-42a6-80a9-318b18cfe4a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6809","nonce":2556929312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6811","nonce":2556929312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6815","nonce":2556929312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":2556929312},{"type":"v1","addr":"172.21.15.152:6813","nonce":2556929312}]},"public_addr":"172.21.15.152:6809/2556929312","cluster_addr":"172.21.15.152:6811/2556929312","heartbeat_back_addr":"172.21.15.152:6815/2556929312","heartbeat_front_addr":"172.21.15.152:6813/2556929312","state":["exists","up"]},{"osd":5,"uuid":"537029e6-890b-47da-973c-c3397db9751e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6809","nonce":2766250253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6811","nonce":2766250253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6815","nonce":2766250253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2766250253},{"type":"v1","addr":"172.21.15.196:6813","nonce":2766250253}]},"public_addr":"172.21.15.196:6809/2766250253","cluster_addr":"172.21.15.196:6811/2766250253","heartbeat_back_addr":"172.21.15.196:6815/2766250253","heartbeat_front_addr":"172.21.15.196:6813/2766250253","state":["exists","up"]},{"osd":6,"uuid":"7803a823-c8f7-4015-ba43-f8c0df7eb053","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6819","nonce":4272451125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6821","nonce":4272451125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6825","nonce":4272451125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4272451125},{"type":"v1","addr":"172.21.15.78:6823","nonce":4272451125}]},"public_addr":"172.21.15.78:6819/4272451125","cluster_addr":"172.21.15.78:6821/4272451125","heartbeat_back_addr":"172.21.15.78:6825/4272451125","heartbeat_front_addr":"172.21.15.78:6823/4272451125","state":["exists","up"]},{"osd":7,"uuid":"9db92c0f-3343-4fdf-b894-f55c250c741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6817","nonce":4200178251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6819","nonce":4200178251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6823","nonce":4200178251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4200178251},{"type":"v1","addr":"172.21.15.152:6821","nonce":4200178251}]},"public_addr":"172.21.15.152:6817/4200178251","cluster_addr":"172.21.15.152:6819/4200178251","heartbeat_back_addr":"172.21.15.152:6823/4200178251","heartbeat_front_addr":"172.21.15.152:6821/4200178251","state":["exists","up"]},{"osd":8,"uuid":"28f5d9f1-e260-43b1-9741-2717211085e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6817","nonce":3733356426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6819","nonce":3733356426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6823","nonce":3733356426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3733356426},{"type":"v1","addr":"172.21.15.196:6821","nonce":3733356426}]},"public_addr":"172.21.15.196:6817/3733356426","cluster_addr":"172.21.15.196:6819/3733356426","heartbeat_back_addr":"172.21.15.196:6823/3733356426","heartbeat_front_addr":"172.21.15.196:6821/3733356426","state":["exists","up"]},{"osd":9,"uuid":"07876526-a50e-4031-ad91-552cd02a1b6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6827","nonce":3327461277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6829","nonce":3327461277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6833","nonce":3327461277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":3327461277},{"type":"v1","addr":"172.21.15.78:6831","nonce":3327461277}]},"public_addr":"172.21.15.78:6827/3327461277","cluster_addr":"172.21.15.78:6829/3327461277","heartbeat_back_addr":"172.21.15.78:6833/3327461277","heartbeat_front_addr":"172.21.15.78:6831/3327461277","state":["exists","up"]},{"osd":10,"uuid":"9a1dd985-c7e3-4dd2-97fd-0ad5f81dc0c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6825","nonce":3363913476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6827","nonce":3363913476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6831","nonce":3363913476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3363913476},{"type":"v1","addr":"172.21.15.152:6829","nonce":3363913476}]},"public_addr":"172.21.15.152:6825/3363913476","cluster_addr":"172.21.15.152:6827/3363913476","heartbeat_back_addr":"172.21.15.152:6831/3363913476","heartbeat_front_addr":"172.21.15.152:6829/3363913476","state":["exists","up"]},{"osd":11,"uuid":"a29b5606-d549-4980-bb67-adaf71ac22bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6825","nonce":3010681986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6827","nonce":3010681986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6831","nonce":3010681986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3010681986},{"type":"v1","addr":"172.21.15.196:6829","nonce":3010681986}]},"public_addr":"172.21.15.196:6825/3010681986","cluster_addr":"172.21.15.196:6827/3010681986","heartbeat_back_addr":"172.21.15.196:6831/3010681986","heartbeat_front_addr":"172.21.15.196:6829/3010681986","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:37:48.696018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:12.522696+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:38:36.378475+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:01.210523+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:26.094094+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:39:51.340079+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:16.269361+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:40:41.469882+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:07.415269+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:33.832139+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:41:59.211058+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:42:26.076975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1},{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"4.5","mappings":[{"from":5,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:6800/1101495738":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/2762765405":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/4170716207":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/1136116795":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6801/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:6800/1544860340":"2024-04-26T01:35:50.507447+0000","172.21.15.78:0/211357528":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/88994848":"2024-04-26T01:36:16.812502+0000","172.21.15.78:0/1196975031":"2024-04-26T01:35:50.507447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:48:06.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.865+0000 7f4cc51af640 1 -- 172.21.15.78:0/3055672038 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4ca80a2b90 msgr2=0x7f4ca80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.865+0000 7f4cc51af640 1 --2- 172.21.15.78:0/3055672038 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4ca80a2b90 0x7f4ca80a5050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4cc01027f0 tx=0x7f4cb800a2d0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.865+0000 7f4cc51af640 1 -- 172.21.15.78:0/3055672038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0102990 msgr2=0x7f4cc0197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.865+0000 7f4cc51af640 1 --2- 172.21.15.78:0/3055672038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0102990 0x7f4cc0197f80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400ebc0 tx=0x7f4cb400c6a0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.865+0000 7f4cc51af640 1 -- 172.21.15.78:0/3055672038 shutdown_connections 2024-04-25T01:48:06.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.865+0000 7f4cc51af640 1 --2- 172.21.15.78:0/3055672038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cc0103350 0x7f4cc019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.865+0000 7f4cc51af640 1 --2- 172.21.15.78:0/3055672038 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f4ca80a2b90 0x7f4ca80a5050 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.865+0000 7f4cc51af640 1 --2- 172.21.15.78:0/3055672038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4cc0102990 0x7f4cc0197f80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.865+0000 7f4cc51af640 1 --2- 172.21.15.78:0/3055672038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc01017a0 0x7f4cc0197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.865+0000 7f4cc51af640 1 -- 172.21.15.78:0/3055672038 >> 172.21.15.78:0/3055672038 conn(0x7f4cc00fccf0 msgr2=0x7f4cc0105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:06.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.865+0000 7f4cc51af640 1 -- 172.21.15.78:0/3055672038 shutdown_connections 2024-04-25T01:48:06.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.865+0000 7f4cc51af640 1 -- 172.21.15.78:0/3055672038 wait complete. 2024-04-25T01:48:06.874 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T01:48:06.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.976+0000 7febabad4640 1 -- 172.21.15.78:0/2023542769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feba4103340 msgr2=0x7feba4109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.976+0000 7febabad4640 1 --2- 172.21.15.78:0/2023542769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feba4103340 0x7feba4109bd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009ca0 tx=0x7feb9c02a8c0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.976+0000 7febabad4640 1 -- 172.21.15.78:0/2023542769 shutdown_connections 2024-04-25T01:48:06.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.976+0000 7febabad4640 1 --2- 172.21.15.78:0/2023542769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feba4103340 0x7feba4109bd0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.976+0000 7febabad4640 1 --2- 172.21.15.78:0/2023542769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feba4102980 0x7feba4102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.976+0000 7febabad4640 1 --2- 172.21.15.78:0/2023542769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feba4101790 0x7feba4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.976+0000 7febabad4640 1 -- 172.21.15.78:0/2023542769 >> 172.21.15.78:0/2023542769 conn(0x7feba40fcd20 msgr2=0x7feba40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:06.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.976+0000 7febabad4640 1 -- 172.21.15.78:0/2023542769 shutdown_connections 2024-04-25T01:48:06.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.976+0000 7febabad4640 1 -- 172.21.15.78:0/2023542769 wait complete. 2024-04-25T01:48:06.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febabad4640 1 Processor -- start 2024-04-25T01:48:06.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febabad4640 1 -- start start 2024-04-25T01:48:06.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febabad4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feba4101790 0x7feba4197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:06.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febabad4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feba4102980 0x7feba4197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:06.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febabad4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feba4103340 0x7feba419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:06.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febabad4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feba410d410 con 0x7feba4103340 2024-04-25T01:48:06.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febabad4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7feba410d290 con 0x7feba4101790 2024-04-25T01:48:06.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febabad4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feba410d590 con 0x7feba4102980 2024-04-25T01:48:06.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febaa071640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feba4103340 0x7feba419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:06.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febaa071640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feba4103340 0x7feba419efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57222/0 (socket says 172.21.15.78:57222) 2024-04-25T01:48:06.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febaa071640 1 -- 172.21.15.78:0/1409688667 learned_addr learned my addr 172.21.15.78:0/1409688667 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:48:06.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7feba9870640 1 --2- 172.21.15.78:0/1409688667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feba4101790 0x7feba4197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:06.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7feba906f640 1 --2- 172.21.15.78:0/1409688667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feba4102980 0x7feba4197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:06.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febaa071640 1 -- 172.21.15.78:0/1409688667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feba4102980 msgr2=0x7feba4197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febaa071640 1 --2- 172.21.15.78:0/1409688667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feba4102980 0x7feba4197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febaa071640 1 -- 172.21.15.78:0/1409688667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feba4101790 msgr2=0x7feba4197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febaa071640 1 --2- 172.21.15.78:0/1409688667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feba4101790 0x7feba4197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.977+0000 7febaa071640 1 -- 172.21.15.78:0/1409688667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba419f640 con 0x7feba4103340 2024-04-25T01:48:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.978+0000 7febaa071640 1 --2- 172.21.15.78:0/1409688667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feba4103340 0x7feba419efd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7feb9c005e30 tx=0x7feb9c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.978+0000 7feb9affd640 1 -- 172.21.15.78:0/1409688667 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c003c30 con 0x7feba4103340 2024-04-25T01:48:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.978+0000 7feb9affd640 1 -- 172.21.15.78:0/1409688667 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb9c003dd0 con 0x7feba4103340 2024-04-25T01:48:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.978+0000 7feb9affd640 1 -- 172.21.15.78:0/1409688667 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c02fde0 con 0x7feba4103340 2024-04-25T01:48:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.978+0000 7febabad4640 1 -- 172.21.15.78:0/1409688667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba419f8d0 con 0x7feba4103340 2024-04-25T01:48:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.978+0000 7febabad4640 1 -- 172.21.15.78:0/1409688667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba419fe20 con 0x7feba4103340 2024-04-25T01:48:06.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.979+0000 7febabad4640 1 -- 172.21.15.78:0/1409688667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba4108a50 con 0x7feba4103340 2024-04-25T01:48:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.980+0000 7feb9affd640 1 -- 172.21.15.78:0/1409688667 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7feb9c004050 con 0x7feba4103340 2024-04-25T01:48:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.980+0000 7feb9affd640 1 --2- 172.21.15.78:0/1409688667 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7feb900a2b90 0x7feb900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.980+0000 7feba9870640 1 --2- 172.21.15.78:0/1409688667 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7feb900a2b90 0x7feb900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.981+0000 7feb9affd640 1 -- 172.21.15.78:0/1409688667 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9224+0+0 (secure 0 0 0) 0x7feb9c0f0680 con 0x7feba4103340 2024-04-25T01:48:06.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.981+0000 7feba9870640 1 --2- 172.21.15.78:0/1409688667 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7feb900a2b90 0x7feb900a5050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feba0005fd0 tx=0x7feba000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:06.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:06.985+0000 7feb9affd640 1 -- 172.21.15.78:0/1409688667 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feb9c0bd200 con 0x7feba4103340 2024-04-25T01:48:07.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.198+0000 7febabad4640 1 -- 172.21.15.78:0/1409688667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7feba4062340 con 0x7feba4103340 2024-04-25T01:48:07.196 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:07 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/4283543837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:48:07.196 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:07 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/737832893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:48:07.196 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:07 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3055672038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:48:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.201+0000 7feb9affd640 1 -- 172.21.15.78:0/1409688667 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7feb9c0bcb00 con 0x7feba4103340 2024-04-25T01:48:07.197 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T01:48:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.205+0000 7febabad4640 1 -- 172.21.15.78:0/1409688667 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7feb900a2b90 msgr2=0x7feb900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.205+0000 7febabad4640 1 --2- 172.21.15.78:0/1409688667 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7feb900a2b90 0x7feb900a5050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feba0005fd0 tx=0x7feba000b390 comp rx=0 tx=0).stop 2024-04-25T01:48:07.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.205+0000 7febabad4640 1 -- 172.21.15.78:0/1409688667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feba4103340 msgr2=0x7feba419efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.205+0000 7febabad4640 1 --2- 172.21.15.78:0/1409688667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feba4103340 0x7feba419efd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7feb9c005e30 tx=0x7feb9c031ee0 comp rx=0 tx=0).stop 2024-04-25T01:48:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.205+0000 7febabad4640 1 -- 172.21.15.78:0/1409688667 shutdown_connections 2024-04-25T01:48:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.206+0000 7febabad4640 1 --2- 172.21.15.78:0/1409688667 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7feb900a2b90 0x7feb900a5050 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.206+0000 7febabad4640 1 --2- 172.21.15.78:0/1409688667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feba4103340 0x7feba419efd0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.206+0000 7febabad4640 1 --2- 172.21.15.78:0/1409688667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feba4102980 0x7feba4197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.206+0000 7febabad4640 1 --2- 172.21.15.78:0/1409688667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feba4101790 0x7feba4197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.206+0000 7febabad4640 1 -- 172.21.15.78:0/1409688667 >> 172.21.15.78:0/1409688667 conn(0x7feba40fcd20 msgr2=0x7feba4105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.206+0000 7febabad4640 1 -- 172.21.15.78:0/1409688667 shutdown_connections 2024-04-25T01:48:07.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.206+0000 7febabad4640 1 -- 172.21.15.78:0/1409688667 wait complete. 2024-04-25T01:48:07.214 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T01:48:07.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/4283543837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:48:07.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/737832893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:48:07.303 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3055672038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:48:07.303 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-h[62501]: ignoring --setuser ceph since I am not root 2024-04-25T01:48:07.303 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-h[62501]: ignoring --setgroup ceph since I am not root 2024-04-25T01:48:07.304 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-mds[62505]: -- 172.21.15.152:0/615328490 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561782d8fa40 con 0x561782d62800 2024-04-25T01:48:07.304 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-h[62501]: starting mds.h at 2024-04-25T01:48:07.304 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-b[62804]: ignoring --setuser ceph since I am not root 2024-04-25T01:48:07.304 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-b[62804]: ignoring --setgroup ceph since I am not root 2024-04-25T01:48:07.304 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-mds[62808]: -- 172.21.15.152:0/2459966994 <== mon.2 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56293cfa7a40 con 0x56293cf7a800 2024-04-25T01:48:07.304 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-b[62804]: starting mds.b at 2024-04-25T01:48:07.304 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-k[61860]: ignoring --setuser ceph since I am not root 2024-04-25T01:48:07.305 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-k[61860]: ignoring --setgroup ceph since I am not root 2024-04-25T01:48:07.305 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-mds[61864]: -- 172.21.15.152:0/2061964698 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f07bb7fa40 con 0x55f07bb52800 2024-04-25T01:48:07.305 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-k[61860]: starting mds.k at 2024-04-25T01:48:07.305 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-e[62178]: ignoring --setuser ceph since I am not root 2024-04-25T01:48:07.305 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-e[62178]: ignoring --setgroup ceph since I am not root 2024-04-25T01:48:07.305 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-mds[62182]: -- 172.21.15.152:0/246050853 <== mon.2 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56149f4a3a40 con 0x56149f477000 2024-04-25T01:48:07.305 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-e[62178]: starting mds.e at 2024-04-25T01:48:07.305 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:48:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-e[62178]: 2024-04-25T01:48:07.250+0000 7fa3678a4700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6834/0: (98) Address already in use 2024-04-25T01:48:07.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.339+0000 7ff4f27a9640 1 -- 172.21.15.78:0/4246791880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4ec100680 msgr2=0x7ff4ec100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:07.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.339+0000 7ff4f27a9640 1 --2- 172.21.15.78:0/4246791880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4ec100680 0x7ff4ec100aa0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009900 tx=0x7ff4dc02a410 comp rx=0 tx=0).stop 2024-04-25T01:48:07.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 -- 172.21.15.78:0/4246791880 shutdown_connections 2024-04-25T01:48:07.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 --2- 172.21.15.78:0/4246791880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4ec102230 0x7ff4ec108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:07.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 --2- 172.21.15.78:0/4246791880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ec101870 0x7ff4ec101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:07.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 --2- 172.21.15.78:0/4246791880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4ec100680 0x7ff4ec100aa0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:07.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 -- 172.21.15.78:0/4246791880 >> 172.21.15.78:0/4246791880 conn(0x7ff4ec0fbbf0 msgr2=0x7ff4ec0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:07.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 -- 172.21.15.78:0/4246791880 shutdown_connections 2024-04-25T01:48:07.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 -- 172.21.15.78:0/4246791880 wait complete. 2024-04-25T01:48:07.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 Processor -- start 2024-04-25T01:48:07.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 -- start start 2024-04-25T01:48:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4ec100680 0x7ff4ec1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4ec101870 0x7ff4ec102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ec102230 0x7ff4ec103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec10c2d0 con 0x7ff4ec100680 2024-04-25T01:48:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec10c150 con 0x7ff4ec101870 2024-04-25T01:48:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.340+0000 7ff4f27a9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec10c450 con 0x7ff4ec102230 2024-04-25T01:48:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4ebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4ec100680 0x7ff4ec1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4ebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4ec100680 0x7ff4ec1045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57268/0 (socket says 172.21.15.78:57268) 2024-04-25T01:48:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4ebfff640 1 -- 172.21.15.78:0/1107747067 learned_addr learned my addr 172.21.15.78:0/1107747067 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:48:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4eb7fe640 1 --2- 172.21.15.78:0/1107747067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4ec101870 0x7ff4ec102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4f0d46640 1 --2- 172.21.15.78:0/1107747067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ec102230 0x7ff4ec103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4ebfff640 1 -- 172.21.15.78:0/1107747067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ec102230 msgr2=0x7ff4ec103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4ebfff640 1 --2- 172.21.15.78:0/1107747067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ec102230 0x7ff4ec103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:07.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4ebfff640 1 -- 172.21.15.78:0/1107747067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4ec101870 msgr2=0x7ff4ec102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:07.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4ebfff640 1 --2- 172.21.15.78:0/1107747067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4ec101870 0x7ff4ec102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:07.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4ebfff640 1 -- 172.21.15.78:0/1107747067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ec103a50 con 0x7ff4ec100680 2024-04-25T01:48:07.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4ebfff640 1 --2- 172.21.15.78:0/1107747067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4ec100680 0x7ff4ec1045c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc031610 tx=0x7ff4dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:07.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4e97fa640 1 -- 172.21.15.78:0/1107747067 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc03f030 con 0x7ff4ec100680 2024-04-25T01:48:07.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4e97fa640 1 -- 172.21.15.78:0/1107747067 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4dc003ad0 con 0x7ff4ec100680 2024-04-25T01:48:07.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4f27a9640 1 -- 172.21.15.78:0/1107747067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ec1a1b90 con 0x7ff4ec100680 2024-04-25T01:48:07.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4e97fa640 1 -- 172.21.15.78:0/1107747067 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc035070 con 0x7ff4ec100680 2024-04-25T01:48:07.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.341+0000 7ff4f27a9640 1 -- 172.21.15.78:0/1107747067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ec1a20a0 con 0x7ff4ec100680 2024-04-25T01:48:07.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.342+0000 7ff4f27a9640 1 -- 172.21.15.78:0/1107747067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ec1a2330 con 0x7ff4ec100680 2024-04-25T01:48:07.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.344+0000 7ff4e97fa640 1 -- 172.21.15.78:0/1107747067 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7ff4dc042030 con 0x7ff4ec100680 2024-04-25T01:48:07.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.345+0000 7ff4e97fa640 1 --2- 172.21.15.78:0/1107747067 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff4d40a2c10 0x7ff4d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:07.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.345+0000 7ff4e97fa640 1 -- 172.21.15.78:0/1107747067 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9654+0+0 (secure 0 0 0) 0x7ff4dc0efee0 con 0x7ff4ec100680 2024-04-25T01:48:07.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.345+0000 7ff4eb7fe640 1 --2- 172.21.15.78:0/1107747067 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff4d40a2c10 0x7ff4d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:07.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.345+0000 7ff4eb7fe640 1 --2- 172.21.15.78:0/1107747067 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff4d40a2c10 0x7ff4d40a50d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec104330 tx=0x7ff4e000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:07.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.348+0000 7ff4e97fa640 1 -- 172.21.15.78:0/1107747067 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff4dc0bc8b0 con 0x7ff4ec100680 2024-04-25T01:48:07.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/4283543837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:48:07.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/737832893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:48:07.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3055672038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:48:07.371 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-c[63966]: ignoring --setuser ceph since I am not root 2024-04-25T01:48:07.371 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-c[63966]: ignoring --setgroup ceph since I am not root 2024-04-25T01:48:07.371 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-mds[63970]: -- 172.21.15.196:0/3498103478 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556f031624e0 con 0x556f03139000 2024-04-25T01:48:07.371 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-mds[63970]: -- 172.21.15.196:0/3498103478 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f03165a40 con 0x556f03139000 2024-04-25T01:48:07.371 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-c[63966]: starting mds.c at 2024-04-25T01:48:07.372 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-l[63352]: ignoring --setuser ceph since I am not root 2024-04-25T01:48:07.372 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-l[63352]: ignoring --setgroup ceph since I am not root 2024-04-25T01:48:07.372 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-mds[63356]: -- 172.21.15.196:0/4082830282 <== mon.2 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d095d7fa40 con 0x55d095d52800 2024-04-25T01:48:07.372 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-l[63352]: starting mds.l at 2024-04-25T01:48:07.372 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-i[64283]: ignoring --setuser ceph since I am not root 2024-04-25T01:48:07.372 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-i[64283]: ignoring --setgroup ceph since I am not root 2024-04-25T01:48:07.373 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-mds[64287]: -- 172.21.15.196:0/698510337 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fcd5c2fa40 con 0x55fcd5c03000 2024-04-25T01:48:07.373 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-i[64283]: starting mds.i at 2024-04-25T01:48:07.373 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-f[63658]: ignoring --setuser ceph since I am not root 2024-04-25T01:48:07.373 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-f[63658]: ignoring --setgroup ceph since I am not root 2024-04-25T01:48:07.373 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-mds[63662]: -- 172.21.15.196:0/1622233244 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5557e48b7a40 con 0x5557e488ac00 2024-04-25T01:48:07.373 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:48:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-f[63658]: starting mds.f at 2024-04-25T01:48:07.499 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:07 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a[29465]: 2024-04-25T01:48:07.199+0000 7f28988e4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:48:07.499 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:48:07 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-a[68467]: ignoring --setuser ceph since I am not root 2024-04-25T01:48:07.499 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:48:07 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-a[68467]: ignoring --setgroup ceph since I am not root 2024-04-25T01:48:07.499 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:48:07 smithi078 ceph-mds[68471]: -- 172.21.15.78:0/652262767 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5650fef77a40 con 0x5650fef4ac00 2024-04-25T01:48:07.499 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:48:07 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-a[68467]: starting mds.a at 2024-04-25T01:48:07.500 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:48:07 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-g[68108]: ignoring --setuser ceph since I am not root 2024-04-25T01:48:07.500 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:48:07 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-g[68108]: ignoring --setgroup ceph since I am not root 2024-04-25T01:48:07.500 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:48:07 smithi078 ceph-mds[68112]: -- 172.21.15.78:0/65758535 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5586fb0e1a40 con 0x5586fb0b4800 2024-04-25T01:48:07.500 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:48:07 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-g[68108]: starting mds.g at 2024-04-25T01:48:07.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:07.569+0000 7ff4f27a9640 1 -- 172.21.15.78:0/1107747067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff4ec1a2500 con 0x7ff4ec100680 2024-04-25T01:48:08.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.202+0000 7ff4e97fa640 1 -- 172.21.15.78:0/1107747067 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7ff4dc0bc1b0 con 0x7ff4ec100680 2024-04-25T01:48:08.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.205+0000 7ff4f27a9640 1 -- 172.21.15.78:0/1107747067 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff4d40a2c10 msgr2=0x7ff4d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:08.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.205+0000 7ff4f27a9640 1 --2- 172.21.15.78:0/1107747067 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff4d40a2c10 0x7ff4d40a50d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec104330 tx=0x7ff4e000a2d0 comp rx=0 tx=0).stop 2024-04-25T01:48:08.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.205+0000 7ff4f27a9640 1 -- 172.21.15.78:0/1107747067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4ec100680 msgr2=0x7ff4ec1045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:08.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.205+0000 7ff4f27a9640 1 --2- 172.21.15.78:0/1107747067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4ec100680 0x7ff4ec1045c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc031610 tx=0x7ff4dc004270 comp rx=0 tx=0).stop 2024-04-25T01:48:08.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.205+0000 7ff4f27a9640 1 -- 172.21.15.78:0/1107747067 shutdown_connections 2024-04-25T01:48:08.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.205+0000 7ff4f27a9640 1 --2- 172.21.15.78:0/1107747067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4ec102230 0x7ff4ec103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:08.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.205+0000 7ff4f27a9640 1 --2- 172.21.15.78:0/1107747067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4ec101870 0x7ff4ec102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:08.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.205+0000 7ff4f27a9640 1 --2- 172.21.15.78:0/1107747067 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7ff4d40a2c10 0x7ff4d40a50d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:08.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.205+0000 7ff4f27a9640 1 --2- 172.21.15.78:0/1107747067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4ec100680 0x7ff4ec1045c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:08.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.205+0000 7ff4f27a9640 1 -- 172.21.15.78:0/1107747067 >> 172.21.15.78:0/1107747067 conn(0x7ff4ec0fbbf0 msgr2=0x7ff4ec0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:08.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.205+0000 7ff4f27a9640 1 -- 172.21.15.78:0/1107747067 shutdown_connections 2024-04-25T01:48:08.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.205+0000 7ff4f27a9640 1 -- 172.21.15.78:0/1107747067 wait complete. 2024-04-25T01:48:08.216 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T01:48:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.326+0000 7f47a4c7b640 1 -- 172.21.15.78:0/2388915574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0102980 msgr2=0x7f47a0102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.326+0000 7f47a4c7b640 1 --2- 172.21.15.78:0/2388915574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0102980 0x7f47a0102e00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4794009900 tx=0x7f479402a410 comp rx=0 tx=0).stop 2024-04-25T01:48:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.326+0000 7f47a4c7b640 1 -- 172.21.15.78:0/2388915574 shutdown_connections 2024-04-25T01:48:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.326+0000 7f47a4c7b640 1 --2- 172.21.15.78:0/2388915574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a0103340 0x7f47a0109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.326+0000 7f47a4c7b640 1 --2- 172.21.15.78:0/2388915574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0102980 0x7f47a0102e00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.326+0000 7f47a4c7b640 1 --2- 172.21.15.78:0/2388915574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47a0101790 0x7f47a0101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.326+0000 7f47a4c7b640 1 -- 172.21.15.78:0/2388915574 >> 172.21.15.78:0/2388915574 conn(0x7f47a00fcd20 msgr2=0x7f47a00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.326+0000 7f47a4c7b640 1 -- 172.21.15.78:0/2388915574 shutdown_connections 2024-04-25T01:48:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.326+0000 7f47a4c7b640 1 -- 172.21.15.78:0/2388915574 wait complete. 2024-04-25T01:48:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f47a4c7b640 1 Processor -- start 2024-04-25T01:48:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f47a4c7b640 1 -- start start 2024-04-25T01:48:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f47a4c7b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47a0101790 0x7f47a0197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f47a4c7b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0102980 0x7f47a0198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f47a4c7b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a0103340 0x7f47a019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f47a4c7b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47a010d410 con 0x7f47a0102980 2024-04-25T01:48:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f47a4c7b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f47a010d290 con 0x7f47a0103340 2024-04-25T01:48:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f47a4c7b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f47a010d590 con 0x7f47a0101790 2024-04-25T01:48:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f479dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0102980 0x7f47a0198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f479dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0102980 0x7f47a0198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57280/0 (socket says 172.21.15.78:57280) 2024-04-25T01:48:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f479dd9b640 1 -- 172.21.15.78:0/1026050718 learned_addr learned my addr 172.21.15.78:0/1026050718 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:48:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f479e59c640 1 --2- 172.21.15.78:0/1026050718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47a0101790 0x7f47a0197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f479ed9d640 1 --2- 172.21.15.78:0/1026050718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a0103340 0x7f47a019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f479dd9b640 1 -- 172.21.15.78:0/1026050718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47a0101790 msgr2=0x7f47a0197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f479dd9b640 1 --2- 172.21.15.78:0/1026050718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47a0101790 0x7f47a0197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f479dd9b640 1 -- 172.21.15.78:0/1026050718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a0103340 msgr2=0x7f47a019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.327+0000 7f479dd9b640 1 --2- 172.21.15.78:0/1026050718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a0103340 0x7f47a019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.328+0000 7f479dd9b640 1 -- 172.21.15.78:0/1026050718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a019f700 con 0x7f47a0102980 2024-04-25T01:48:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.328+0000 7f479dd9b640 1 --2- 172.21.15.78:0/1026050718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0102980 0x7f47a0198020 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4794009230 tx=0x7f4794004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.328+0000 7f478f7fe640 1 -- 172.21.15.78:0/1026050718 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479403f030 con 0x7f47a0102980 2024-04-25T01:48:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.328+0000 7f47a4c7b640 1 -- 172.21.15.78:0/1026050718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a019f990 con 0x7f47a0102980 2024-04-25T01:48:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.328+0000 7f478f7fe640 1 -- 172.21.15.78:0/1026050718 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4794003ce0 con 0x7f47a0102980 2024-04-25T01:48:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.328+0000 7f478f7fe640 1 -- 172.21.15.78:0/1026050718 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794035070 con 0x7f47a0102980 2024-04-25T01:48:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.328+0000 7f47a4c7b640 1 -- 172.21.15.78:0/1026050718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a019fe80 con 0x7f47a0102980 2024-04-25T01:48:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.329+0000 7f47a4c7b640 1 -- 172.21.15.78:0/1026050718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a0108b50 con 0x7f47a0102980 2024-04-25T01:48:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.330+0000 7f478f7fe640 1 -- 172.21.15.78:0/1026050718 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7f4794042030 con 0x7f47a0102980 2024-04-25T01:48:08.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.331+0000 7f478f7fe640 1 --2- 172.21.15.78:0/1026050718 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f47880a2b90 0x7f47880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:08.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.331+0000 7f479e59c640 1 --2- 172.21.15.78:0/1026050718 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f47880a2b90 0x7f47880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:08.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.331+0000 7f478f7fe640 1 -- 172.21.15.78:0/1026050718 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9654+0+0 (secure 0 0 0) 0x7f47940f0060 con 0x7f47a0102980 2024-04-25T01:48:08.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.331+0000 7f479e59c640 1 --2- 172.21.15.78:0/1026050718 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f47880a2b90 0x7f47880a5050 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4790005fd0 tx=0x7f4790005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:08.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.339+0000 7f478f7fe640 1 -- 172.21.15.78:0/1026050718 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f47940bca30 con 0x7f47a0102980 2024-04-25T01:48:08.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:08 smithi196 ceph-mon[32978]: pgmap v439: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T01:48:08.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:08 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1409688667' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T01:48:08.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:08 smithi196 ceph-mon[32978]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T01:48:08.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:08 smithi196 ceph-mon[32978]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:48:08.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:08 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1409688667' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T01:48:08.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:08 smithi196 ceph-mon[32978]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T01:48:08.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:08 smithi196 ceph-mon[32978]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T01:48:08.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:08 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1107747067' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:48:08.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:08 smithi078 ceph-mon[29469]: pgmap v439: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T01:48:08.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:08 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1409688667' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T01:48:08.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:08 smithi078 ceph-mon[29469]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T01:48:08.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:08 smithi078 ceph-mon[29469]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:48:08.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:08 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1409688667' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T01:48:08.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:08 smithi078 ceph-mon[29469]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T01:48:08.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:08 smithi078 ceph-mon[29469]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T01:48:08.443 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:08 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1107747067' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:48:08.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:08.520+0000 7f47a4c7b640 1 -- 172.21.15.78:0/1026050718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f47a002d330 con 0x7f47a0102980 2024-04-25T01:48:08.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:08 smithi152 ceph-mon[32465]: pgmap v439: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T01:48:08.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:08 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1409688667' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T01:48:08.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:08 smithi152 ceph-mon[32465]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T01:48:08.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:08 smithi152 ceph-mon[32465]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:48:08.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:08 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1409688667' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T01:48:08.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:08 smithi152 ceph-mon[32465]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T01:48:08.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:08 smithi152 ceph-mon[32465]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T01:48:08.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:08 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1107747067' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:48:09.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.204+0000 7f478f7fe640 1 -- 172.21.15.78:0/1026050718 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7f47a002d330 con 0x7f47a0102980 2024-04-25T01:48:09.201 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-04-25T01:48:09.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.207+0000 7f47a4c7b640 1 -- 172.21.15.78:0/1026050718 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f47880a2b90 msgr2=0x7f47880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:09.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.207+0000 7f47a4c7b640 1 --2- 172.21.15.78:0/1026050718 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f47880a2b90 0x7f47880a5050 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4790005fd0 tx=0x7f4790005850 comp rx=0 tx=0).stop 2024-04-25T01:48:09.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.207+0000 7f47a4c7b640 1 -- 172.21.15.78:0/1026050718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0102980 msgr2=0x7f47a0198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:09.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.207+0000 7f47a4c7b640 1 --2- 172.21.15.78:0/1026050718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0102980 0x7f47a0198020 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4794009230 tx=0x7f4794004360 comp rx=0 tx=0).stop 2024-04-25T01:48:09.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.207+0000 7f47a4c7b640 1 -- 172.21.15.78:0/1026050718 shutdown_connections 2024-04-25T01:48:09.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.207+0000 7f47a4c7b640 1 --2- 172.21.15.78:0/1026050718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a0103340 0x7f47a019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:09.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.207+0000 7f47a4c7b640 1 --2- 172.21.15.78:0/1026050718 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7f47880a2b90 0x7f47880a5050 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:09.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.207+0000 7f47a4c7b640 1 --2- 172.21.15.78:0/1026050718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a0102980 0x7f47a0198020 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:09.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.207+0000 7f47a4c7b640 1 --2- 172.21.15.78:0/1026050718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47a0101790 0x7f47a0197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:09.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.207+0000 7f47a4c7b640 1 -- 172.21.15.78:0/1026050718 >> 172.21.15.78:0/1026050718 conn(0x7f47a00fcd20 msgr2=0x7f47a0105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:09.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.208+0000 7f47a4c7b640 1 -- 172.21.15.78:0/1026050718 shutdown_connections 2024-04-25T01:48:09.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.208+0000 7f47a4c7b640 1 -- 172.21.15.78:0/1026050718 wait complete. 2024-04-25T01:48:09.218 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T01:48:09.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.328+0000 7efc5e19e640 1 -- 172.21.15.78:0/3734124871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc58101890 msgr2=0x7efc58101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:09.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.328+0000 7efc5e19e640 1 --2- 172.21.15.78:0/3734124871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc58101890 0x7efc58101d10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7efc40009900 tx=0x7efc4002a410 comp rx=0 tx=0).stop 2024-04-25T01:48:09.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.328+0000 7efc5e19e640 1 -- 172.21.15.78:0/3734124871 shutdown_connections 2024-04-25T01:48:09.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.328+0000 7efc5e19e640 1 --2- 172.21.15.78:0/3734124871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc58102250 0x7efc58108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:09.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.328+0000 7efc5e19e640 1 --2- 172.21.15.78:0/3734124871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc58101890 0x7efc58101d10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:09.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.328+0000 7efc5e19e640 1 --2- 172.21.15.78:0/3734124871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc581006a0 0x7efc58100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.328+0000 7efc5e19e640 1 -- 172.21.15.78:0/3734124871 >> 172.21.15.78:0/3734124871 conn(0x7efc580fbc30 msgr2=0x7efc580fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.328+0000 7efc5e19e640 1 -- 172.21.15.78:0/3734124871 shutdown_connections 2024-04-25T01:48:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.328+0000 7efc5e19e640 1 -- 172.21.15.78:0/3734124871 wait complete. 2024-04-25T01:48:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc5e19e640 1 Processor -- start 2024-04-25T01:48:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc5e19e640 1 -- start start 2024-04-25T01:48:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc5e19e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc581006a0 0x7efc581969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc5e19e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc58101890 0x7efc58196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc5e19e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc58102250 0x7efc5819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc5e19e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc5810c320 con 0x7efc581006a0 2024-04-25T01:48:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc5e19e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efc5810c1a0 con 0x7efc58101890 2024-04-25T01:48:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc5e19e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efc5810c4a0 con 0x7efc58102250 2024-04-25T01:48:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc581006a0 0x7efc581969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc581006a0 0x7efc581969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57302/0 (socket says 172.21.15.78:57302) 2024-04-25T01:48:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc577fe640 1 -- 172.21.15.78:0/2470609003 learned_addr learned my addr 172.21.15.78:0/2470609003 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:48:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc4ffff640 1 --2- 172.21.15.78:0/2470609003 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc58101890 0x7efc58196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc577fe640 1 -- 172.21.15.78:0/2470609003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc58102250 msgr2=0x7efc5819df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc577fe640 1 --2- 172.21.15.78:0/2470609003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc58102250 0x7efc5819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc577fe640 1 -- 172.21.15.78:0/2470609003 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc58101890 msgr2=0x7efc58196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc577fe640 1 --2- 172.21.15.78:0/2470609003 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc58101890 0x7efc58196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc577fe640 1 -- 172.21.15.78:0/2470609003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5819e5e0 con 0x7efc581006a0 2024-04-25T01:48:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc577fe640 1 --2- 172.21.15.78:0/2470609003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc581006a0 0x7efc581969c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7efc4800d770 tx=0x7efc4800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc557fa640 1 -- 172.21.15.78:0/2470609003 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48017070 con 0x7efc581006a0 2024-04-25T01:48:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc557fa640 1 -- 172.21.15.78:0/2470609003 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc48009c30 con 0x7efc581006a0 2024-04-25T01:48:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc5e19e640 1 -- 172.21.15.78:0/2470609003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5819e8d0 con 0x7efc581006a0 2024-04-25T01:48:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.329+0000 7efc557fa640 1 -- 172.21.15.78:0/2470609003 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48003dd0 con 0x7efc581006a0 2024-04-25T01:48:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.330+0000 7efc5e19e640 1 -- 172.21.15.78:0/2470609003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5819ee40 con 0x7efc581006a0 2024-04-25T01:48:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.330+0000 7efc5e19e640 1 -- 172.21.15.78:0/2470609003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc581079c0 con 0x7efc581006a0 2024-04-25T01:48:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.331+0000 7efc557fa640 1 -- 172.21.15.78:0/2470609003 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7efc48003fb0 con 0x7efc581006a0 2024-04-25T01:48:09.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.332+0000 7efc557fa640 1 --2- 172.21.15.78:0/2470609003 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efc440a2c10 0x7efc440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:09.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.332+0000 7efc4ffff640 1 --2- 172.21.15.78:0/2470609003 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efc440a2c10 0x7efc440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:09.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.332+0000 7efc557fa640 1 -- 172.21.15.78:0/2470609003 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9092+0+0 (secure 0 0 0) 0x7efc480cede0 con 0x7efc581006a0 2024-04-25T01:48:09.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.333+0000 7efc4ffff640 1 --2- 172.21.15.78:0/2470609003 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efc440a2c10 0x7efc440a50d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efc40000c00 tx=0x7efc400315d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:09.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.337+0000 7efc557fa640 1 -- 172.21.15.78:0/2470609003 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efc48009de0 con 0x7efc581006a0 2024-04-25T01:48:09.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T01:48:09.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T01:48:09.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: Cluster is now healthy 2024-04-25T01:48:09.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1107747067' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T01:48:09.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.78:6838/2583163381,v1:172.21.15.78:6840/2583163381] up:boot 2024-04-25T01:48:09.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.78:6839/3037424475,v1:172.21.15.78:6841/3037424475] up:boot 2024-04-25T01:48:09.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.152:6833/3224177268,v1:172.21.15.152:6835/3224177268] up:boot 2024-04-25T01:48:09.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.152:6832/1402295134,v1:172.21.15.152:6834/1402295134] up:boot 2024-04-25T01:48:09.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.196:6832/2977678991,v1:172.21.15.196:6833/2977678991] up:boot 2024-04-25T01:48:09.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.196:6834/2053443158,v1:172.21.15.196:6835/2053443158] up:boot 2024-04-25T01:48:09.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.152:6836/1672586605,v1:172.21.15.152:6837/1672586605] up:boot 2024-04-25T01:48:09.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.152:6838/2976799016,v1:172.21.15.152:6839/2976799016] up:boot 2024-04-25T01:48:09.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.196:6836/1919470215,v1:172.21.15.196:6837/1919470215] up:boot 2024-04-25T01:48:09.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: mds.? [v2:172.21.15.196:6838/3166426037,v1:172.21.15.196:6839/3166426037] up:boot 2024-04-25T01:48:09.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: fsmap 12 up:standby 2024-04-25T01:48:09.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T01:48:09.372 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T01:48:09.373 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T01:48:09.373 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T01:48:09.373 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T01:48:09.373 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T01:48:09.373 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T01:48:09.373 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T01:48:09.373 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T01:48:09.373 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T01:48:09.373 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:09 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1026050718' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:48:09.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T01:48:09.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T01:48:09.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: Cluster is now healthy 2024-04-25T01:48:09.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1107747067' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T01:48:09.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.78:6838/2583163381,v1:172.21.15.78:6840/2583163381] up:boot 2024-04-25T01:48:09.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.78:6839/3037424475,v1:172.21.15.78:6841/3037424475] up:boot 2024-04-25T01:48:09.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.152:6833/3224177268,v1:172.21.15.152:6835/3224177268] up:boot 2024-04-25T01:48:09.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.152:6832/1402295134,v1:172.21.15.152:6834/1402295134] up:boot 2024-04-25T01:48:09.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.196:6832/2977678991,v1:172.21.15.196:6833/2977678991] up:boot 2024-04-25T01:48:09.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.196:6834/2053443158,v1:172.21.15.196:6835/2053443158] up:boot 2024-04-25T01:48:09.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.152:6836/1672586605,v1:172.21.15.152:6837/1672586605] up:boot 2024-04-25T01:48:09.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.152:6838/2976799016,v1:172.21.15.152:6839/2976799016] up:boot 2024-04-25T01:48:09.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.196:6836/1919470215,v1:172.21.15.196:6837/1919470215] up:boot 2024-04-25T01:48:09.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: mds.? [v2:172.21.15.196:6838/3166426037,v1:172.21.15.196:6839/3166426037] up:boot 2024-04-25T01:48:09.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: fsmap 12 up:standby 2024-04-25T01:48:09.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T01:48:09.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T01:48:09.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T01:48:09.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T01:48:09.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T01:48:09.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T01:48:09.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T01:48:09.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T01:48:09.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T01:48:09.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T01:48:09.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:09 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1026050718' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:48:09.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:09.523+0000 7efc5e19e640 1 -- 172.21.15.78:0/2470609003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7efc5802d170 con 0x7efc581006a0 2024-04-25T01:48:09.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T01:48:09.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T01:48:09.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: Cluster is now healthy 2024-04-25T01:48:09.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1107747067' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T01:48:09.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.78:6838/2583163381,v1:172.21.15.78:6840/2583163381] up:boot 2024-04-25T01:48:09.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.78:6839/3037424475,v1:172.21.15.78:6841/3037424475] up:boot 2024-04-25T01:48:09.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.152:6833/3224177268,v1:172.21.15.152:6835/3224177268] up:boot 2024-04-25T01:48:09.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.152:6832/1402295134,v1:172.21.15.152:6834/1402295134] up:boot 2024-04-25T01:48:09.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.196:6832/2977678991,v1:172.21.15.196:6833/2977678991] up:boot 2024-04-25T01:48:09.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.196:6834/2053443158,v1:172.21.15.196:6835/2053443158] up:boot 2024-04-25T01:48:09.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.152:6836/1672586605,v1:172.21.15.152:6837/1672586605] up:boot 2024-04-25T01:48:09.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.152:6838/2976799016,v1:172.21.15.152:6839/2976799016] up:boot 2024-04-25T01:48:09.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.196:6836/1919470215,v1:172.21.15.196:6837/1919470215] up:boot 2024-04-25T01:48:09.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: mds.? [v2:172.21.15.196:6838/3166426037,v1:172.21.15.196:6839/3166426037] up:boot 2024-04-25T01:48:09.554 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: fsmap 12 up:standby 2024-04-25T01:48:09.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T01:48:09.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T01:48:09.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T01:48:09.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T01:48:09.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T01:48:09.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T01:48:09.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T01:48:09.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T01:48:09.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T01:48:09.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: from='mgr.14148 172.21.15.78:0/4175100980' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T01:48:09.555 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:09 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1026050718' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:48:10.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.212+0000 7efc557fa640 1 -- 172.21.15.78:0/2470609003 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7efc4809b9e0 con 0x7efc581006a0 2024-04-25T01:48:10.208 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-04-25T01:48:10.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.215+0000 7efc5e19e640 1 -- 172.21.15.78:0/2470609003 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efc440a2c10 msgr2=0x7efc440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:10.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.215+0000 7efc5e19e640 1 --2- 172.21.15.78:0/2470609003 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efc440a2c10 0x7efc440a50d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efc40000c00 tx=0x7efc400315d0 comp rx=0 tx=0).stop 2024-04-25T01:48:10.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.215+0000 7efc5e19e640 1 -- 172.21.15.78:0/2470609003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc581006a0 msgr2=0x7efc581969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:10.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.215+0000 7efc5e19e640 1 --2- 172.21.15.78:0/2470609003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc581006a0 0x7efc581969c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7efc4800d770 tx=0x7efc4800dc30 comp rx=0 tx=0).stop 2024-04-25T01:48:10.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.215+0000 7efc5e19e640 1 -- 172.21.15.78:0/2470609003 shutdown_connections 2024-04-25T01:48:10.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.215+0000 7efc5e19e640 1 --2- 172.21.15.78:0/2470609003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc58102250 0x7efc5819df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:10.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.215+0000 7efc5e19e640 1 --2- 172.21.15.78:0/2470609003 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc58101890 0x7efc58196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:10.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.215+0000 7efc5e19e640 1 --2- 172.21.15.78:0/2470609003 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7efc440a2c10 0x7efc440a50d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:10.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.215+0000 7efc5e19e640 1 --2- 172.21.15.78:0/2470609003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc581006a0 0x7efc581969c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:10.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.215+0000 7efc5e19e640 1 -- 172.21.15.78:0/2470609003 >> 172.21.15.78:0/2470609003 conn(0x7efc580fbc30 msgr2=0x7efc58104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:10.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.216+0000 7efc5e19e640 1 -- 172.21.15.78:0/2470609003 shutdown_connections 2024-04-25T01:48:10.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.216+0000 7efc5e19e640 1 -- 172.21.15.78:0/2470609003 wait complete. 2024-04-25T01:48:10.228 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T01:48:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.338+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/1623176584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a80ffa50 msgr2=0x7fc0a80ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.338+0000 7fc0ad0a7640 1 --2- 172.21.15.78:0/1623176584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a80ffa50 0x7fc0a80ffe70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc098009930 tx=0x7fc09802a420 comp rx=0 tx=0).stop 2024-04-25T01:48:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.338+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/1623176584 shutdown_connections 2024-04-25T01:48:10.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.338+0000 7fc0ad0a7640 1 --2- 172.21.15.78:0/1623176584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0a81033e0 0x7fc0a8109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.338+0000 7fc0ad0a7640 1 --2- 172.21.15.78:0/1623176584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0a8102a20 0x7fc0a8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.338+0000 7fc0ad0a7640 1 --2- 172.21.15.78:0/1623176584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a80ffa50 0x7fc0a80ffe70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.338+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/1623176584 >> 172.21.15.78:0/1623176584 conn(0x7fc0a80fb380 msgr2=0x7fc0a80fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.338+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/1623176584 shutdown_connections 2024-04-25T01:48:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.338+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/1623176584 wait complete. 2024-04-25T01:48:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0ad0a7640 1 Processor -- start 2024-04-25T01:48:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0ad0a7640 1 -- start start 2024-04-25T01:48:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0ad0a7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0a80ffa50 0x7fc0a8197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0ad0a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a8102a20 0x7fc0a8198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0ad0a7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0a81033e0 0x7fc0a819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:10.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0ad0a7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a810d480 con 0x7fc0a8102a20 2024-04-25T01:48:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0ad0a7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a810d300 con 0x7fc0a80ffa50 2024-04-25T01:48:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0ad0a7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a810d600 con 0x7fc0a81033e0 2024-04-25T01:48:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0a659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a8102a20 0x7fc0a8198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0a659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a8102a20 0x7fc0a8198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44104/0 (socket says 172.21.15.78:44104) 2024-04-25T01:48:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0a659c640 1 -- 172.21.15.78:0/3058590882 learned_addr learned my addr 172.21.15.78:0/3058590882 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-04-25T01:48:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0a6d9d640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0a80ffa50 0x7fc0a8197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0a759e640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0a81033e0 0x7fc0a819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0a659c640 1 -- 172.21.15.78:0/3058590882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0a81033e0 msgr2=0x7fc0a819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0a659c640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0a81033e0 0x7fc0a819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:10.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0a659c640 1 -- 172.21.15.78:0/3058590882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0a80ffa50 msgr2=0x7fc0a8197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0a659c640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0a80ffa50 0x7fc0a8197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0a659c640 1 -- 172.21.15.78:0/3058590882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a819f770 con 0x7fc0a8102a20 2024-04-25T01:48:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0a759e640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0a81033e0 0x7fc0a819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:48:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.339+0000 7fc0a6d9d640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0a80ffa50 0x7fc0a8197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:48:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.340+0000 7fc0a659c640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a8102a20 0x7fc0a8198090 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00c890 tx=0x7fc09c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.340+0000 7fc08ffff640 1 -- 172.21.15.78:0/3058590882 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c004220 con 0x7fc0a8102a20 2024-04-25T01:48:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.340+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/3058590882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a819fa60 con 0x7fc0a8102a20 2024-04-25T01:48:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.340+0000 7fc08ffff640 1 -- 172.21.15.78:0/3058590882 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc09c0043c0 con 0x7fc0a8102a20 2024-04-25T01:48:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.340+0000 7fc08ffff640 1 -- 172.21.15.78:0/3058590882 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c004a00 con 0x7fc0a8102a20 2024-04-25T01:48:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.340+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/3058590882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a819ffb0 con 0x7fc0a8102a20 2024-04-25T01:48:10.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.341+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/3058590882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a8108bd0 con 0x7fc0a8102a20 2024-04-25T01:48:10.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.342+0000 7fc08ffff640 1 -- 172.21.15.78:0/3058590882 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 137833+0+0 (secure 0 0 0) 0x7fc09c0079b0 con 0x7fc0a8102a20 2024-04-25T01:48:10.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.343+0000 7fc08ffff640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc0900a2b90 0x7fc0900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:10.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.343+0000 7fc0a6d9d640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc0900a2b90 0x7fc0900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:10.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.343+0000 7fc08ffff640 1 -- 172.21.15.78:0/3058590882 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8496+0+0 (secure 0 0 0) 0x7fc09c0cf2f0 con 0x7fc0a8102a20 2024-04-25T01:48:10.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.343+0000 7fc0a6d9d640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc0900a2b90 0x7fc0900a5050 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc098009900 tx=0x7fc098000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:10.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.346+0000 7fc08ffff640 1 -- 172.21.15.78:0/3058590882 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc09c014030 con 0x7fc0a8102a20 2024-04-25T01:48:10.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:10 smithi196 ceph-mon[32978]: pgmap v441: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-25T01:48:10.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:10 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/1026050718' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T01:48:10.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:10 smithi196 ceph-mon[32978]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T01:48:10.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:10 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2470609003' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:48:10.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:10 smithi078 ceph-mon[29469]: pgmap v441: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-25T01:48:10.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:10 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/1026050718' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T01:48:10.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:10 smithi078 ceph-mon[29469]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T01:48:10.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:10 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2470609003' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:48:10.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:10.537+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/3058590882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc0a802d170 con 0x7fc0a8102a20 2024-04-25T01:48:10.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:10 smithi152 ceph-mon[32465]: pgmap v441: 129 pgs: 129 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-25T01:48:10.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:10 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/1026050718' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T01:48:10.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:10 smithi152 ceph-mon[32465]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T01:48:10.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:10 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2470609003' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:48:11.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:11.217+0000 7fc08ffff640 1 -- 172.21.15.78:0/3058590882 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) ==== 178+0+0 (secure 0 0 0) 0x7fc09c0ce0d0 con 0x7fc0a8102a20 2024-04-25T01:48:11.214 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data_ec' removed 2024-04-25T01:48:11.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:11.221+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/3058590882 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc0900a2b90 msgr2=0x7fc0900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:11.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:11.221+0000 7fc0ad0a7640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc0900a2b90 0x7fc0900a5050 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc098009900 tx=0x7fc098000bc0 comp rx=0 tx=0).stop 2024-04-25T01:48:11.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:11.221+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/3058590882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a8102a20 msgr2=0x7fc0a8198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:11.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:11.221+0000 7fc0ad0a7640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a8102a20 0x7fc0a8198090 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00c890 tx=0x7fc09c00cd50 comp rx=0 tx=0).stop 2024-04-25T01:48:11.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:11.221+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/3058590882 shutdown_connections 2024-04-25T01:48:11.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:11.221+0000 7fc0ad0a7640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0a81033e0 0x7fc0a819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:11.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:11.221+0000 7fc0ad0a7640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.78:6800/2190983290,v1:172.21.15.78:6801/2190983290] conn(0x7fc0900a2b90 0x7fc0900a5050 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:11.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:11.221+0000 7fc0ad0a7640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a8102a20 0x7fc0a8198090 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:11.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:11.221+0000 7fc0ad0a7640 1 --2- 172.21.15.78:0/3058590882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0a80ffa50 0x7fc0a8197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:11.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:11.221+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/3058590882 >> 172.21.15.78:0/3058590882 conn(0x7fc0a80fb380 msgr2=0x7fc0a8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:11.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:11.221+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/3058590882 shutdown_connections 2024-04-25T01:48:11.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-25T01:48:11.222+0000 7fc0ad0a7640 1 -- 172.21.15.78:0/3058590882 wait complete. 2024-04-25T01:48:11.234 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:48:11.260 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:48:11.295 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:48:11.324 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T01:48:11.324 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T01:48:11.340 DEBUG:teuthology.orchestra.run.smithi152:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T01:48:11.357 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T01:48:11.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:11 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/2470609003' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T01:48:11.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:11 smithi196 ceph-mon[32978]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T01:48:11.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:11 smithi196 ceph-mon[32978]: from='client.? 172.21.15.78:0/3058590882' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:48:11.371 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:11 smithi196 ceph-mon[32978]: pgmap v444: 33 pgs: 33 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:48:11.380 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T01:48:11.380 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T01:48:11.380 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.a 2024-04-25T01:48:11.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:11 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/2470609003' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T01:48:11.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:11 smithi078 ceph-mon[29469]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T01:48:11.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:11 smithi078 ceph-mon[29469]: from='client.? 172.21.15.78:0/3058590882' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:48:11.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:11 smithi078 ceph-mon[29469]: pgmap v444: 33 pgs: 33 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:48:11.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:11 smithi078 systemd[1]: Stopping Ceph mon.a for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:48:11.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:11 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/2470609003' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T01:48:11.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:11 smithi152 ceph-mon[32465]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T01:48:11.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:11 smithi152 ceph-mon[32465]: from='client.? 172.21.15.78:0/3058590882' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T01:48:11.553 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:11 smithi152 ceph-mon[32465]: pgmap v444: 33 pgs: 33 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T01:48:12.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:11 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a[29465]: 2024-04-25T01:48:11.663+0000 7f289e0ef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:48:12.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:11 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a[29465]: 2024-04-25T01:48:11.663+0000 7f289e0ef700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T01:48:12.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:11 smithi078 podman[83856]: 2024-04-25 01:48:11.735783377 +0000 UTC m=+0.169773689 container died c83da7f37b2e468370b9b4712e1debf357ad9c2fb4b2bb3eeea92ab0628e9004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T01:48:12.454 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:12 smithi078 podman[83856]: 2024-04-25 01:48:12.207323477 +0000 UTC m=+0.641313785 container cleanup c83da7f37b2e468370b9b4712e1debf357ad9c2fb4b2bb3eeea92ab0628e9004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T01:48:12.455 INFO:journalctl@ceph.mon.a.smithi078.stdout:Apr 25 01:48:12 smithi078 bash[83856]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-a 2024-04-25T01:48:12.555 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T01:48:12.611 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:48:12.611 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T01:48:12.611 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T01:48:12.612 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.b 2024-04-25T01:48:12.921 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:12 smithi152 systemd[1]: Stopping Ceph mon.b for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:48:13.305 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:12 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-b[32457]: 2024-04-25T01:48:12.925+0000 7f7e5e3ff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:48:13.306 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:12 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-b[32457]: 2024-04-25T01:48:12.925+0000 7f7e5e3ff700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T01:48:13.306 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:13 smithi152 podman[65719]: 2024-04-25 01:48:13.037770736 +0000 UTC m=+0.230163325 container died 5608e8e66536bf7f2c4225dd19c241f068623fb2652ca728e23fbae13510faeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-25T01:48:13.791 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:13 smithi152 podman[65719]: 2024-04-25 01:48:13.510507912 +0000 UTC m=+0.702900507 container cleanup 5608e8e66536bf7f2c4225dd19c241f068623fb2652ca728e23fbae13510faeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T01:48:13.791 INFO:journalctl@ceph.mon.b.smithi152.stdout:Apr 25 01:48:13 smithi152 bash[65719]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-b 2024-04-25T01:48:13.931 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T01:48:13.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:48:13.981 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T01:48:13.981 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T01:48:13.981 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.c 2024-04-25T01:48:14.294 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:14 smithi196 systemd[1]: Stopping Ceph mon.c for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:48:14.621 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:14 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-c[32974]: 2024-04-25T01:48:14.297+0000 7f9a6d31f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:48:14.621 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:14 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-c[32974]: 2024-04-25T01:48:14.297+0000 7f9a6d31f700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T01:48:14.621 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:14 smithi196 podman[67318]: 2024-04-25 01:48:14.502597951 +0000 UTC m=+0.323517236 container died 7fe86deb75641ed4e3c62ea9b1dd149624dff2ac60b5c768be442e691a47c846 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-04-25T01:48:15.121 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:14 smithi196 podman[67318]: 2024-04-25 01:48:14.931366989 +0000 UTC m=+0.752286322 container cleanup 7fe86deb75641ed4e3c62ea9b1dd149624dff2ac60b5c768be442e691a47c846 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T01:48:15.121 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:14 smithi196 bash[67318]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-c 2024-04-25T01:48:15.403 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T01:48:15.423 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:15 smithi196 podman[67332]: 2024-04-25 01:48:15.174801674 +0000 UTC m=+0.666895395 container remove 7fe86deb75641ed4e3c62ea9b1dd149624dff2ac60b5c768be442e691a47c846 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:48:15.424 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:15 smithi196 systemd[1]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-25T01:48:15.424 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:15 smithi196 systemd[1]: Stopped Ceph mon.c for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:48:15.424 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 25 01:48:15 smithi196 systemd[1]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mon.c.service: Consumed 8.513s CPU time. 2024-04-25T01:48:15.451 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:48:15.451 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T01:48:15.451 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T01:48:15.451 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.x 2024-04-25T01:48:15.761 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:48:15 smithi078 systemd[1]: Stopping Ceph mgr.x for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:48:16.011 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:48:15 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x[29700]: 2024-04-25T01:48:15.761+0000 7f428625f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:48:16.011 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:48:15 smithi078 podman[83976]: 2024-04-25 01:48:15.793288578 +0000 UTC m=+0.144403084 container died 20678b9d90e14170a3ec1565ac720f9405a418229ef573ad6aaaacea445d0720 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:48:16.419 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:48:16 smithi078 podman[83976]: 2024-04-25 01:48:16.238286281 +0000 UTC m=+0.589400787 container cleanup 20678b9d90e14170a3ec1565ac720f9405a418229ef573ad6aaaacea445d0720 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T01:48:16.419 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:48:16 smithi078 bash[83976]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x 2024-04-25T01:48:16.654 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T01:48:16.675 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:48:16 smithi078 podman[83990]: 2024-04-25 01:48:16.423586443 +0000 UTC m=+0.629774221 container remove 20678b9d90e14170a3ec1565ac720f9405a418229ef573ad6aaaacea445d0720 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T01:48:16.676 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:48:16 smithi078 systemd[1]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.x.service: Deactivated successfully. 2024-04-25T01:48:16.676 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:48:16 smithi078 systemd[1]: Stopped Ceph mgr.x for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:48:16.676 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Apr 25 01:48:16 smithi078 systemd[1]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.x.service: Consumed 56.453s CPU time. 2024-04-25T01:48:16.690 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:48:16.690 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T01:48:16.690 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T01:48:16.690 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.y 2024-04-25T01:48:17.006 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:48:16 smithi152 systemd[1]: Stopping Ceph mgr.y for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:48:17.303 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:48:17 smithi152 podman[65841]: 2024-04-25 01:48:17.011244189 +0000 UTC m=+0.127836188 container died 270917cafea24cb7a6a59480a80fb218068bc49f60f80b38e2149ecb739b81fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T01:48:17.671 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:48:17 smithi152 podman[65841]: 2024-04-25 01:48:17.391515945 +0000 UTC m=+0.508107937 container cleanup 270917cafea24cb7a6a59480a80fb218068bc49f60f80b38e2149ecb739b81fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y, RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T01:48:17.672 INFO:journalctl@ceph.mgr.y.smithi152.stdout:Apr 25 01:48:17 smithi152 bash[65841]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-y 2024-04-25T01:48:17.810 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T01:48:17.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:48:17.844 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T01:48:17.844 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T01:48:17.845 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.z 2024-04-25T01:48:18.121 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:48:17 smithi196 systemd[1]: Stopping Ceph mgr.z for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:48:18.508 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:48:18 smithi196 podman[67439]: 2024-04-25 01:48:18.190249724 +0000 UTC m=+0.133462390 container died ab555bd66c8c1ac920c1ab0bb174999b9cb5d807ab517f1c6dc0eb059f5b16a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T01:48:18.871 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:48:18 smithi196 podman[67439]: 2024-04-25 01:48:18.612200271 +0000 UTC m=+0.555412929 container cleanup ab555bd66c8c1ac920c1ab0bb174999b9cb5d807ab517f1c6dc0eb059f5b16a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-04-25T01:48:18.871 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 25 01:48:18 smithi196 bash[67439]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mgr-z 2024-04-25T01:48:19.177 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T01:48:19.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:48:19.210 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T01:48:19.210 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T01:48:19.210 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.0 2024-04-25T01:48:19.505 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:48:19 smithi078 systemd[1]: Stopping Ceph osd.0 for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:48:19.761 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:48:19 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-0[41942]: 2024-04-25T01:48:19.508+0000 7f9de863a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:48:19.761 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:48:19 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-0[41942]: 2024-04-25T01:48:19.508+0000 7f9de863a700 -1 osd.0 98 *** Got signal Terminated *** 2024-04-25T01:48:19.761 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:48:19 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-0[41942]: 2024-04-25T01:48:19.508+0000 7f9de863a700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:48:25.323 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:48:25 smithi078 podman[84098]: 2024-04-25 01:48:25.075524593 +0000 UTC m=+5.678486702 container died 0fa2caa4a2ca86abe098a08f0ab8533627ae9b5b0185c8f65748dc81031f3d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T01:48:25.683 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:48:25 smithi078 podman[84098]: 2024-04-25 01:48:25.494120105 +0000 UTC m=+6.097082268 container cleanup 0fa2caa4a2ca86abe098a08f0ab8533627ae9b5b0185c8f65748dc81031f3d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-0, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T01:48:25.683 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:48:25 smithi078 bash[84098]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-0 2024-04-25T01:48:26.011 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:48:25 smithi078 podman[84116]: 2024-04-25 01:48:25.687701301 +0000 UTC m=+0.610727256 container remove 0fa2caa4a2ca86abe098a08f0ab8533627ae9b5b0185c8f65748dc81031f3d3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T01:48:26.280 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:48:26 smithi078 podman[84207]: 2024-04-25 01:48:26.012666424 +0000 UTC m=+0.018088343 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:48:26.280 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:48:26 smithi078 podman[84207]: 2024-04-25 01:48:26.163818108 +0000 UTC m=+0.169240019 container create 9565405bb0a868614c5c67e911924b53ced6ea7bd90fbd7d8b90e6c181e0882b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T01:48:26.683 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:48:26 smithi078 podman[84207]: 2024-04-25 01:48:26.396352234 +0000 UTC m=+0.401774148 container init 9565405bb0a868614c5c67e911924b53ced6ea7bd90fbd7d8b90e6c181e0882b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T01:48:26.683 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:48:26 smithi078 podman[84207]: 2024-04-25 01:48:26.399730578 +0000 UTC m=+0.405152490 container start 9565405bb0a868614c5c67e911924b53ced6ea7bd90fbd7d8b90e6c181e0882b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T01:48:26.683 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:48:26 smithi078 podman[84207]: 2024-04-25 01:48:26.500068847 +0000 UTC m=+0.505490773 container attach 9565405bb0a868614c5c67e911924b53ced6ea7bd90fbd7d8b90e6c181e0882b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T01:48:27.011 INFO:journalctl@ceph.osd.0.smithi078.stdout:Apr 25 01:48:26 smithi078 podman[84207]: 2024-04-25 01:48:26.687643215 +0000 UTC m=+0.693065131 container died 9565405bb0a868614c5c67e911924b53ced6ea7bd90fbd7d8b90e6c181e0882b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T01:48:27.395 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T01:48:27.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:48:27.433 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T01:48:27.433 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T01:48:27.434 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.1 2024-04-25T01:48:27.750 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:48:27 smithi152 systemd[1]: Stopping Ceph osd.1 for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:48:28.053 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:48:27 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-1[37052]: 2024-04-25T01:48:27.754+0000 7fdfb6969700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:48:28.053 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:48:27 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-1[37052]: 2024-04-25T01:48:27.754+0000 7fdfb6969700 -1 osd.1 98 *** Got signal Terminated *** 2024-04-25T01:48:28.054 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:48:27 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-1[37052]: 2024-04-25T01:48:27.754+0000 7fdfb6969700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:48:33.401 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:48:33 smithi152 podman[65967]: 2024-04-25 01:48:33.1233341 +0000 UTC m=+5.492650416 container died 6f5b16a92481f78cb51f3efcf87659c275c3fc5de421c16ce52437beef894cbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-1, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T01:48:33.786 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:48:33 smithi152 podman[65967]: 2024-04-25 01:48:33.513261115 +0000 UTC m=+5.882577427 container cleanup 6f5b16a92481f78cb51f3efcf87659c275c3fc5de421c16ce52437beef894cbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T01:48:33.786 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:48:33 smithi152 bash[65967]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-1 2024-04-25T01:48:34.053 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:48:33 smithi152 podman[65984]: 2024-04-25 01:48:33.790117457 +0000 UTC m=+0.660262275 container remove 6f5b16a92481f78cb51f3efcf87659c275c3fc5de421c16ce52437beef894cbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T01:48:34.423 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:48:34 smithi152 podman[66077]: 2024-04-25 01:48:34.031356196 +0000 UTC m=+0.017957746 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:48:34.423 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:48:34 smithi152 podman[66077]: 2024-04-25 01:48:34.158089995 +0000 UTC m=+0.144691585 container create 5f1721efbddc23fb316fdf1ae6abc73afea11561827a7381918b9d1621fd0cfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T01:48:34.715 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:48:34 smithi152 podman[66077]: 2024-04-25 01:48:34.427563792 +0000 UTC m=+0.414165335 container init 5f1721efbddc23fb316fdf1ae6abc73afea11561827a7381918b9d1621fd0cfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:48:34.716 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:48:34 smithi152 podman[66077]: 2024-04-25 01:48:34.430660583 +0000 UTC m=+0.417262138 container start 5f1721efbddc23fb316fdf1ae6abc73afea11561827a7381918b9d1621fd0cfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-04-25T01:48:34.716 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:48:34 smithi152 podman[66077]: 2024-04-25 01:48:34.527610107 +0000 UTC m=+0.514211663 container attach 5f1721efbddc23fb316fdf1ae6abc73afea11561827a7381918b9d1621fd0cfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T01:48:35.053 INFO:journalctl@ceph.osd.1.smithi152.stdout:Apr 25 01:48:34 smithi152 podman[66077]: 2024-04-25 01:48:34.720045338 +0000 UTC m=+0.706646887 container died 5f1721efbddc23fb316fdf1ae6abc73afea11561827a7381918b9d1621fd0cfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T01:48:35.419 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T01:48:35.456 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:48:35.457 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T01:48:35.457 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T01:48:35.457 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.2 2024-04-25T01:48:35.745 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:48:35 smithi196 systemd[1]: Stopping Ceph osd.2 for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:48:36.001 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:48:35 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-2[38112]: 2024-04-25T01:48:35.747+0000 7f9e4f0d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:48:36.001 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:48:35 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-2[38112]: 2024-04-25T01:48:35.747+0000 7f9e4f0d1700 -1 osd.2 98 *** Got signal Terminated *** 2024-04-25T01:48:36.001 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:48:35 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-2[38112]: 2024-04-25T01:48:35.747+0000 7f9e4f0d1700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:48:41.488 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:48:41 smithi196 podman[67566]: 2024-04-25 01:48:41.191159647 +0000 UTC m=+5.547525206 container died 6c3c03ecd59b58287b95c427fc9c8dafd1a3476fa193e9c400556f6bee3ef41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-2, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T01:48:41.804 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:48:41 smithi196 podman[67566]: 2024-04-25 01:48:41.590715876 +0000 UTC m=+5.947081438 container cleanup 6c3c03ecd59b58287b95c427fc9c8dafd1a3476fa193e9c400556f6bee3ef41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T01:48:41.805 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:48:41 smithi196 bash[67566]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-2 2024-04-25T01:48:42.121 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:48:41 smithi196 podman[67596]: 2024-04-25 01:48:41.808778025 +0000 UTC m=+0.613995399 container remove 6c3c03ecd59b58287b95c427fc9c8dafd1a3476fa193e9c400556f6bee3ef41c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-25T01:48:42.508 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:48:42 smithi196 podman[67685]: 2024-04-25 01:48:42.108826184 +0000 UTC m=+0.017926351 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:48:42.508 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:48:42 smithi196 podman[67685]: 2024-04-25 01:48:42.259828077 +0000 UTC m=+0.168928227 container create ef1b48c8efcc085be3131d0195c77b71dd891789775b8fab75a425900af00e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T01:48:42.807 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:48:42 smithi196 podman[67685]: 2024-04-25 01:48:42.512570539 +0000 UTC m=+0.421670688 container init ef1b48c8efcc085be3131d0195c77b71dd891789775b8fab75a425900af00e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T01:48:42.808 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:48:42 smithi196 podman[67685]: 2024-04-25 01:48:42.515803249 +0000 UTC m=+0.424903410 container start ef1b48c8efcc085be3131d0195c77b71dd891789775b8fab75a425900af00e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-25T01:48:42.808 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:48:42 smithi196 podman[67685]: 2024-04-25 01:48:42.612847283 +0000 UTC m=+0.521947444 container attach ef1b48c8efcc085be3131d0195c77b71dd891789775b8fab75a425900af00e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T01:48:43.121 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 25 01:48:42 smithi196 podman[67685]: 2024-04-25 01:48:42.812288865 +0000 UTC m=+0.721389023 container died ef1b48c8efcc085be3131d0195c77b71dd891789775b8fab75a425900af00e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T01:48:43.537 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T01:48:43.577 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:48:43.578 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T01:48:43.578 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T01:48:43.578 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.3 2024-04-25T01:48:43.879 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:43 smithi078 systemd[1]: Stopping Ceph osd.3 for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:48:44.261 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:43 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3[48445]: 2024-04-25T01:48:43.881+0000 7fed2a0ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:48:44.262 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:43 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3[48445]: 2024-04-25T01:48:43.881+0000 7fed2a0ab700 -1 osd.3 98 *** Got signal Terminated *** 2024-04-25T01:48:44.262 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:43 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3[48445]: 2024-04-25T01:48:43.881+0000 7fed2a0ab700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:48:46.121 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:45 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:45.744+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:46.554 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:46 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:46.206+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:48:47.121 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:46 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:46.742+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:47.512 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:47 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:47.024+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:48:47.553 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:47 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:47.191+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:48:47.871 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:47 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:47.706+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:47.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:47 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:47.496+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:48:48.262 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:48 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3[48445]: 2024-04-25T01:48:48.003+0000 7fed2669f700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.454276+0000 front 2024-04-25T01:48:22.454304+0000 (oldest deadline 2024-04-25T01:48:47.154475+0000) 2024-04-25T01:48:48.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:48 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:48.039+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:48:48.553 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:48 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:48.214+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:48:48.871 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:48 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:48.506+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:48:48.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:48 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:48.657+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:49.362 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:49 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:49.032+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:48:49.553 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:49 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:49.188+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:48:49.655 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:49 smithi078 podman[84315]: 2024-04-25 01:48:49.366596502 +0000 UTC m=+5.598356944 container died 2432217ecad5ffe333da8da0fc7d599ccd44f2128106850788b84880e637a0e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T01:48:49.871 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:49 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:49.472+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:48:49.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:49 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:49.706+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:50.007 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:49 smithi078 podman[84315]: 2024-04-25 01:48:49.850139615 +0000 UTC m=+6.081900110 container cleanup 2432217ecad5ffe333da8da0fc7d599ccd44f2128106850788b84880e637a0e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T01:48:50.007 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:49 smithi078 bash[84315]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3 2024-04-25T01:48:50.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:50 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:50.010+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:48:50.262 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:50 smithi078 podman[84344]: 2024-04-25 01:48:50.16048692 +0000 UTC m=+0.791724581 container remove 2432217ecad5ffe333da8da0fc7d599ccd44f2128106850788b84880e637a0e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:48:50.554 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:50 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:50.176+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:48:50.768 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:50 smithi078 podman[84436]: 2024-04-25 01:48:50.423228598 +0000 UTC m=+0.017830140 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:48:50.768 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:50 smithi078 podman[84436]: 2024-04-25 01:48:50.53648451 +0000 UTC m=+0.131086050 container create 1c21ba7d0fe202b21248d0b8bd4e2b813d45dc45cca776a37c742feb96748200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-25T01:48:50.769 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:50 smithi078 podman[84436]: 2024-04-25 01:48:50.772435668 +0000 UTC m=+0.367037210 container init 1c21ba7d0fe202b21248d0b8bd4e2b813d45dc45cca776a37c742feb96748200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T01:48:50.871 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:50 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:50.517+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:48:50.871 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:50 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:50.688+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:51.061 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:50 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:50.988+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:48:51.062 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:50 smithi078 podman[84436]: 2024-04-25 01:48:50.775524396 +0000 UTC m=+0.370125946 container start 1c21ba7d0fe202b21248d0b8bd4e2b813d45dc45cca776a37c742feb96748200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-25T01:48:51.062 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:50 smithi078 podman[84436]: 2024-04-25 01:48:50.880937921 +0000 UTC m=+0.475539485 container attach 1c21ba7d0fe202b21248d0b8bd4e2b813d45dc45cca776a37c742feb96748200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T01:48:51.372 INFO:journalctl@ceph.osd.3.smithi078.stdout:Apr 25 01:48:51 smithi078 podman[84436]: 2024-04-25 01:48:51.065919327 +0000 UTC m=+0.660520873 container died 1c21ba7d0fe202b21248d0b8bd4e2b813d45dc45cca776a37c742feb96748200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T01:48:51.553 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:51 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:51.152+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:48:51.811 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T01:48:51.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:48:51.850 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T01:48:51.850 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T01:48:51.851 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.4 2024-04-25T01:48:51.871 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:51 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:51.477+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:48:51.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:51 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:51.710+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:52.156 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:51 smithi152 systemd[1]: Stopping Ceph osd.4 for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:48:52.157 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:52 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:52.129+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:48:52.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:51 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:51.970+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:48:52.554 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:52 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4[43979]: 2024-04-25T01:48:52.159+0000 7f718889d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:48:52.554 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:52 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4[43979]: 2024-04-25T01:48:52.159+0000 7f718889d700 -1 osd.4 98 *** Got signal Terminated *** 2024-04-25T01:48:52.554 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:52 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4[43979]: 2024-04-25T01:48:52.159+0000 7f718889d700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:48:52.871 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:52 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:52.723+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:52.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:52 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:52.519+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:48:53.261 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:52 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:52.949+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:48:53.553 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:53 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:53.137+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:48:53.871 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:53 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:53.492+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:48:53.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:53 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:53.700+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:54.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:53 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:53.945+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:48:54.553 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:54 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:54.120+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:48:54.871 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:54 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:54.537+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:48:54.872 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:48:54 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:48:54.724+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:48:54.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:54 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:54.657+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:54.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:54 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:54.657+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.676858+0000 front 2024-04-25T01:48:32.677032+0000 (oldest deadline 2024-04-25T01:48:53.776805+0000) 2024-04-25T01:48:55.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:54 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:54.929+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:48:55.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:48:54 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:48:54.861+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:48:55.554 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:55 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:55.169+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:48:55.871 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:55 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:55.548+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:48:55.872 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:48:55 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:48:55.731+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:48:55.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:55 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:55.658+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:55.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:55 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:55.658+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.676858+0000 front 2024-04-25T01:48:32.677032+0000 (oldest deadline 2024-04-25T01:48:53.776805+0000) 2024-04-25T01:48:56.151 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:55.923+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:48:56.151 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:48:55 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:48:55.886+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:48:56.214 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:56 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4[43979]: 2024-04-25T01:48:56.134+0000 7f7180e89700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.200768+0000 front 2024-04-25T01:48:31.200967+0000 (oldest deadline 2024-04-25T01:48:55.300957+0000) 2024-04-25T01:48:56.214 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:48:55 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:48:55.934+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:48:56.554 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:56 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:56.217+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:48:56.871 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:48:56 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:48:56.751+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:48:56.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:56 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:56.528+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:48:56.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:56 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:56.641+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:56.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:56 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:56.641+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.676858+0000 front 2024-04-25T01:48:32.677032+0000 (oldest deadline 2024-04-25T01:48:53.776805+0000) 2024-04-25T01:48:57.233 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:57 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4[43979]: 2024-04-25T01:48:57.150+0000 7f7180e89700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.200768+0000 front 2024-04-25T01:48:31.200967+0000 (oldest deadline 2024-04-25T01:48:55.300957+0000) 2024-04-25T01:48:57.233 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:48:56 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:48:56.979+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:48:57.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:56 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:56.963+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:48:57.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:48:56 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:48:56.839+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:48:57.554 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:57 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:57.236+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:48:57.739 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:57 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:57.642+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:57.739 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:57 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:57.642+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.676858+0000 front 2024-04-25T01:48:32.677032+0000 (oldest deadline 2024-04-25T01:48:53.776805+0000) 2024-04-25T01:48:57.740 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:57 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:57.479+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:48:57.740 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:57 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:57.479+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:48:57.857 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:57 smithi152 podman[66194]: 2024-04-25 01:48:57.577252715 +0000 UTC m=+5.528608893 container died 795b0124162980180e408893ffbc86134f8d7c36efcfe7ddc03151ba0917e22d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD) 2024-04-25T01:48:58.121 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:48:57 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:48:57.743+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:48:58.224 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:48:58 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:48:58.017+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:48:58.224 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:57 smithi152 podman[66194]: 2024-04-25 01:48:57.960087022 +0000 UTC m=+5.911443195 container cleanup 795b0124162980180e408893ffbc86134f8d7c36efcfe7ddc03151ba0917e22d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:48:58.224 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:57 smithi152 bash[66194]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4 2024-04-25T01:48:58.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:48:57 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:48:57.888+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:48:58.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:57 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:57.934+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:48:58.554 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:58 smithi152 podman[66211]: 2024-04-25 01:48:58.228127538 +0000 UTC m=+0.653354453 container remove 795b0124162980180e408893ffbc86134f8d7c36efcfe7ddc03151ba0917e22d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T01:48:58.554 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:58 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:58.251+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:48:58.554 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:58 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:58.251+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:48:58.744 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:58 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:58.643+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:58.744 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:58 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:58.643+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.676858+0000 front 2024-04-25T01:48:32.677032+0000 (oldest deadline 2024-04-25T01:48:53.776805+0000) 2024-04-25T01:48:58.744 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:58 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:58.484+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:48:58.744 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:58 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:58.484+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:48:58.836 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:58 smithi152 podman[66301]: 2024-04-25 01:48:58.479388509 +0000 UTC m=+0.018213870 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:48:58.836 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:58 smithi152 podman[66301]: 2024-04-25 01:48:58.604296654 +0000 UTC m=+0.143121995 container create 81adc9781ef0f9fb5fc70e7ca9149877cf5e43fa03ce16957948cb117412c588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T01:48:58.837 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:58 smithi152 podman[66301]: 2024-04-25 01:48:58.840220722 +0000 UTC m=+0.379046060 container init 81adc9781ef0f9fb5fc70e7ca9149877cf5e43fa03ce16957948cb117412c588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T01:48:59.121 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:48:58 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:48:58.747+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:48:59.149 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:48:59 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:48:59.050+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:48:59.149 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:58 smithi152 podman[66301]: 2024-04-25 01:48:58.843284947 +0000 UTC m=+0.382110286 container start 81adc9781ef0f9fb5fc70e7ca9149877cf5e43fa03ce16957948cb117412c588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-25T01:48:59.149 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:58 smithi152 podman[66301]: 2024-04-25 01:48:58.957056397 +0000 UTC m=+0.495881734 container attach 81adc9781ef0f9fb5fc70e7ca9149877cf5e43fa03ce16957948cb117412c588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T01:48:59.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:48:58 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:48:58.847+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:48:59.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:58 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:58.893+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:48:59.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:58 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:58.893+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:48:59.473 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:59 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:59.282+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:48:59.473 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:48:59 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:48:59.282+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:48:59.473 INFO:journalctl@ceph.osd.4.smithi152.stdout:Apr 25 01:48:59 smithi152 podman[66301]: 2024-04-25 01:48:59.153170537 +0000 UTC m=+0.691995878 container died 81adc9781ef0f9fb5fc70e7ca9149877cf5e43fa03ce16957948cb117412c588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T01:48:59.757 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:59 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:59.666+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:48:59.757 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:59 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:48:59.666+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.676858+0000 front 2024-04-25T01:48:32.677032+0000 (oldest deadline 2024-04-25T01:48:53.776805+0000) 2024-04-25T01:48:59.758 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:59 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:59.509+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:48:59.758 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:48:59 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:48:59.509+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:48:59.781 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T01:48:59.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:48:59.821 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T01:48:59.821 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T01:48:59.821 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.5 2024-04-25T01:49:00.122 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:48:59 smithi196 systemd[1]: Stopping Ceph osd.5 for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:49:00.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:48:59 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:48:59.760+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:00.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:48:59 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:48:59.848+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:49:00.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:59 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:59.847+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:00.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:48:59 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:48:59.847+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:00.303 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:00 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:00.047+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:00.304 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:00 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:00.235+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:00.304 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:00 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:00.235+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:00.524 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:00 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:00.194+0000 7f61ef85c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:49:00.524 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:00 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:00.194+0000 7f61ef85c700 -1 osd.5 98 *** Got signal Terminated *** 2024-04-25T01:49:00.524 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:00 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:00.194+0000 7f61ef85c700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:49:00.871 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:00 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:00.761+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:00.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:00 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:00.683+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:49:00.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:00 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:00.683+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.676858+0000 front 2024-04-25T01:48:32.677032+0000 (oldest deadline 2024-04-25T01:48:53.776805+0000) 2024-04-25T01:49:00.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:00 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:00.527+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:00.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:00 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:00.527+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:01.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:00 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:00.875+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:49:01.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:00 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:00.878+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:01.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:00 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:00.878+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:01.554 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:01 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:01.066+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:01.554 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:01 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:01.257+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:01.554 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:01 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:01.257+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:01.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:01 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:01.698+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:49:01.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:01 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:01.698+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.676858+0000 front 2024-04-25T01:48:32.677032+0000 (oldest deadline 2024-04-25T01:48:53.776805+0000) 2024-04-25T01:49:01.872 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:01 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:01.779+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:01.872 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:01 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:01.779+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:01.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:01 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:01.547+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:01.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:01 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:01.547+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:01.873 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:01 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:01.547+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:02.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:01 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:01.848+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:49:02.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:01 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:01.909+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:02.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:01 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:01.909+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:02.299 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:02 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:02.018+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:02.554 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:02 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:02.303+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:02.554 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:02 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:02.303+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:02.826 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:02 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:02.676+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:49:02.826 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:02 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:02.676+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.676858+0000 front 2024-04-25T01:48:32.677032+0000 (oldest deadline 2024-04-25T01:48:53.776805+0000) 2024-04-25T01:49:02.826 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:02 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:02.676+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.877885+0000 front 2024-04-25T01:48:37.877862+0000 (oldest deadline 2024-04-25T01:49:01.978051+0000) 2024-04-25T01:49:02.826 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:02 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:02.536+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:02.827 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:02 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:02.536+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:02.827 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:02 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:02.536+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:03.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:02 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:02.829+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:03.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:02 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:02.829+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:03.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:02 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:02.846+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:49:03.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:02 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:02.846+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.912907+0000 front 2024-04-25T01:48:37.913069+0000 (oldest deadline 2024-04-25T01:49:02.612974+0000) 2024-04-25T01:49:03.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:02 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:02.911+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:03.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:02 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:02.911+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:03.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:02 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:02.911+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:03.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:03 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:03.056+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:03.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:03 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:03.056+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:03.577 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:03 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:03.336+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:03.577 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:03 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:03.336+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:03.787 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:03 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:03.677+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:49:03.787 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:03 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:03.677+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.676858+0000 front 2024-04-25T01:48:32.677032+0000 (oldest deadline 2024-04-25T01:48:53.776805+0000) 2024-04-25T01:49:03.787 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:03 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:03.677+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.877885+0000 front 2024-04-25T01:48:37.877862+0000 (oldest deadline 2024-04-25T01:49:01.978051+0000) 2024-04-25T01:49:03.787 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:03 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:03.532+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:03.787 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:03 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:03.532+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:03.787 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:03 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:03.532+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:04.121 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:03 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:03.790+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:04.121 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:03 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:03.790+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:04.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:03 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:03.876+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:49:04.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:03 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:03.876+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.912907+0000 front 2024-04-25T01:48:37.913069+0000 (oldest deadline 2024-04-25T01:49:02.612974+0000) 2024-04-25T01:49:04.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:03 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:03.926+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:04.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:03 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:03.926+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:04.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:03 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:03.926+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:04.303 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:04 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:04.287+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:04.304 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:04 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:04.287+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:04.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:04 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:04.038+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:04.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:04 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:04.038+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:04.829 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:04 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:04.687+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:24.074413+0000 front 2024-04-25T01:48:24.074508+0000 (oldest deadline 2024-04-25T01:48:45.174474+0000) 2024-04-25T01:49:04.829 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:04 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:04.687+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.676858+0000 front 2024-04-25T01:48:32.677032+0000 (oldest deadline 2024-04-25T01:48:53.776805+0000) 2024-04-25T01:49:04.829 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:04 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5[45016]: 2024-04-25T01:49:04.687+0000 7f61e7e48700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.877885+0000 front 2024-04-25T01:48:37.877862+0000 (oldest deadline 2024-04-25T01:49:01.978051+0000) 2024-04-25T01:49:04.829 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:04 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:04.569+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:04.829 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:04 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:04.569+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:04.829 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:04 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:04.569+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:05.121 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:04 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:04.832+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:05.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:04 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:04.832+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:05.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:04 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:04.836+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:49:05.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:04 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:04.836+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.912907+0000 front 2024-04-25T01:48:37.913069+0000 (oldest deadline 2024-04-25T01:49:02.612974+0000) 2024-04-25T01:49:05.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:04 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:04.935+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:05.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:04 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:04.935+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:05.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:04 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:04.935+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:05.321 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:05 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:05.068+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:05.321 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:05 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:05.068+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:05.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:05 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:05.325+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:05.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:05 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:05.325+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:05.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:05 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:05.325+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:05.872 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:05 smithi196 podman[67793]: 2024-04-25 01:49:05.690468081 +0000 UTC m=+5.670160687 container died 85f1045d5a1dfbaf420064757ba61a298757f2e76a317cd2930dd5a605abcc96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T01:49:05.872 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:05 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:05.791+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:05.872 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:05 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:05.791+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:05.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:05 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:05.608+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:05.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:05 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:05.608+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:05.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:05 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:05.608+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:06.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:05 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:05.883+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:49:06.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:05 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:05.883+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.912907+0000 front 2024-04-25T01:48:37.913069+0000 (oldest deadline 2024-04-25T01:49:02.612974+0000) 2024-04-25T01:49:06.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:05 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:05.929+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:06.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:05 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:05.929+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:06.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:05 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:05.929+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:06.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:06 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:06.027+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:06.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:06 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:06.027+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:06.622 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:06 smithi196 podman[67793]: 2024-04-25 01:49:06.127628121 +0000 UTC m=+6.107320736 container cleanup 85f1045d5a1dfbaf420064757ba61a298757f2e76a317cd2930dd5a605abcc96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T01:49:06.622 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:06 smithi196 bash[67793]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5 2024-04-25T01:49:06.622 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:06 smithi196 podman[67810]: 2024-04-25 01:49:06.345755495 +0000 UTC m=+0.648918238 container remove 85f1045d5a1dfbaf420064757ba61a298757f2e76a317cd2930dd5a605abcc96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T01:49:06.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:06 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:06.339+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:06.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:06 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:06.339+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:06.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:06 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:06.339+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:06.957 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:06 smithi196 podman[67911]: 2024-04-25 01:49:06.695952167 +0000 UTC m=+0.017737457 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:49:06.957 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:06 smithi196 podman[67911]: 2024-04-25 01:49:06.838315695 +0000 UTC m=+0.160100973 container create a4dcfcd271e9258cb7a82756417281f34adf2b350512008dfabe2ef0a02cbc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T01:49:06.958 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:06 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:06.626+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:06.958 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:06 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:06.626+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:06.958 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:06 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:06.626+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:06.959 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:06 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:06.800+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:06.959 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:06 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:06.800+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:07.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:06 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:06.899+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:49:07.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:06 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:06.899+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.912907+0000 front 2024-04-25T01:48:37.913069+0000 (oldest deadline 2024-04-25T01:49:02.612974+0000) 2024-04-25T01:49:07.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:06 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:06.945+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:07.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:06 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:06.945+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:07.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:06 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:06.945+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:07.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:07.051+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:07.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:07.051+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:07.365 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:07 smithi196 podman[67911]: 2024-04-25 01:49:07.074155131 +0000 UTC m=+0.395940408 container init a4dcfcd271e9258cb7a82756417281f34adf2b350512008dfabe2ef0a02cbc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T01:49:07.365 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:07 smithi196 podman[67911]: 2024-04-25 01:49:07.077435736 +0000 UTC m=+0.399221024 container start a4dcfcd271e9258cb7a82756417281f34adf2b350512008dfabe2ef0a02cbc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:49:07.365 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:07 smithi196 podman[67911]: 2024-04-25 01:49:07.174279353 +0000 UTC m=+0.496064651 container attach a4dcfcd271e9258cb7a82756417281f34adf2b350512008dfabe2ef0a02cbc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9) 2024-04-25T01:49:07.365 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 25 01:49:07 smithi196 podman[67911]: 2024-04-25 01:49:07.368785324 +0000 UTC m=+0.690570604 container died a4dcfcd271e9258cb7a82756417281f34adf2b350512008dfabe2ef0a02cbc8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T01:49:07.621 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:07.593+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:07.622 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:07.593+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:07.622 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:07.594+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:07.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:07.348+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:07.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:07.348+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:07.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:07 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:07.348+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:08.058 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:07.838+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:08.059 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:07 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:07.838+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:08.086 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T01:49:08.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:49:08.129 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T01:49:08.129 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T01:49:08.129 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.6 2024-04-25T01:49:08.155 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:07 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:07.891+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:49:08.155 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:07 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:07.891+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.912907+0000 front 2024-04-25T01:48:37.913069+0000 (oldest deadline 2024-04-25T01:49:02.612974+0000) 2024-04-25T01:49:08.155 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:07 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:07.939+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:08.155 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:07 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:07.939+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:08.156 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:07 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:07.939+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:08.339 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:08 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:08.069+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:08.340 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:08 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:08.069+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:08.512 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:08 smithi078 systemd[1]: Stopping Ceph osd.6 for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:49:08.512 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:08 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:08.406+0000 7ffbdbcf6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:49:08.512 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:08 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:08.406+0000 7ffbdbcf6700 -1 osd.6 98 *** Got signal Terminated *** 2024-04-25T01:49:08.513 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:08 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:08.406+0000 7ffbdbcf6700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:49:08.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:08 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:08.342+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:08.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:08 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:08.342+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:08.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:08 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:08.342+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:09.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:08 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:08.636+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:09.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:08 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:08.636+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:09.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:08 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:08.636+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:09.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:08 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:08.789+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:09.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:08 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:08.789+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:09.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:08 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:08.843+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:49:09.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:08 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:08.843+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.912907+0000 front 2024-04-25T01:48:37.913069+0000 (oldest deadline 2024-04-25T01:49:02.612974+0000) 2024-04-25T01:49:09.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:08 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:08.939+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:09.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:08 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:08.939+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:09.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:08 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:08.939+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:09.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:09 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:09.039+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:09.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:09 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:09.039+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:09.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:09 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:09.344+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:09.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:09 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:09.344+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:09.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:09 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:09.344+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:09.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:09 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:09.600+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:09.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:09 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:09.600+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:09.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:09 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:09.600+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:09.872 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:09 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:09.801+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:09.872 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:09 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:09.801+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:10.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:09 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:09.823+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:49:10.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:09 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:09.823+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.912907+0000 front 2024-04-25T01:48:37.913069+0000 (oldest deadline 2024-04-25T01:49:02.612974+0000) 2024-04-25T01:49:10.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:09 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:09.918+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:10.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:09 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:09.918+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:10.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:09 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:09.918+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:10.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:09 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:09.918+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:10.306 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:10 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:10.005+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:10.306 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:10 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:10.005+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:10.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:10 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:10.357+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:10.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:10 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:10.357+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:10.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:10 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:10.357+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:10.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:10 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:10.621+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:10.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:10 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:10.621+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:10.872 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:10 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:10.621+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:10.872 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:10 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:10.808+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:10.872 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:10 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:10.808+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:11.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:10 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:10.837+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:49:11.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:10 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:10.837+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.912907+0000 front 2024-04-25T01:48:37.913069+0000 (oldest deadline 2024-04-25T01:49:02.612974+0000) 2024-04-25T01:49:11.263 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:10 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:10.837+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:46.714000+0000 front 2024-04-25T01:48:46.713914+0000 (oldest deadline 2024-04-25T01:49:10.814014+0000) 2024-04-25T01:49:11.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:10 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:10.871+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:11.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:10 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:10.871+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:11.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:10 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:10.871+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:11.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:10 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:10.871+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:11.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:10 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:10.961+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:11.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:10 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:10.961+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:11.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:11 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:11.319+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:11.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:11 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:11.319+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:11.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:11 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:11.319+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:11.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:11 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:11.319+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:12.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:11 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:11.646+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:12.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:11 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:11.646+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:12.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:11 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:11.646+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:12.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:11 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:11.786+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:12.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:11 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:11.786+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:12.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:11 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:11.821+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:49:12.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:11 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:11.821+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.912907+0000 front 2024-04-25T01:48:37.913069+0000 (oldest deadline 2024-04-25T01:49:02.612974+0000) 2024-04-25T01:49:12.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:11 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:11.821+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:46.714000+0000 front 2024-04-25T01:48:46.713914+0000 (oldest deadline 2024-04-25T01:49:10.814014+0000) 2024-04-25T01:49:12.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:11 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:11.880+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:12.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:11 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:11.880+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:12.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:11 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:11.880+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:12.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:11 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:11.880+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:12.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:12 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:12.008+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:12.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:12 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:12.008+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:12.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:12 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:12.336+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:12.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:12 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:12.336+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:12.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:12 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:12.336+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:12.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:12 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:12.336+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:13.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:12 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:12.805+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:13.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:12 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:12.805+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:13.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:12 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:12.805+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:13.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:12 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:12.689+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:13.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:12 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:12.689+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:13.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:12 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:12.689+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:13.163 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:12 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:12.863+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.411645+0000 front 2024-04-25T01:48:30.411522+0000 (oldest deadline 2024-04-25T01:48:53.911598+0000) 2024-04-25T01:49:13.163 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:12 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:12.863+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.912907+0000 front 2024-04-25T01:48:37.913069+0000 (oldest deadline 2024-04-25T01:49:02.612974+0000) 2024-04-25T01:49:13.164 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:12 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6[55456]: 2024-04-25T01:49:12.863+0000 7ffbd42e2700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:46.714000+0000 front 2024-04-25T01:48:46.713914+0000 (oldest deadline 2024-04-25T01:49:10.814014+0000) 2024-04-25T01:49:13.164 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:12 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:12.882+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:13.164 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:12 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:12.882+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:13.164 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:12 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:12.882+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:13.164 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:12 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:12.882+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:13.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:13 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:13.050+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:13.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:13 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:13.050+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:13.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:13 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:13.050+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.586461+0000 front 2024-04-25T01:48:48.586480+0000 (oldest deadline 2024-04-25T01:49:12.686307+0000) 2024-04-25T01:49:13.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:13 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:13.357+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:13.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:13 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:13.357+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:13.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:13 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:13.357+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:13.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:13 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:13.357+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:14.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:13 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:13.853+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:14.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:13 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:13.853+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:14.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:13 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:13.853+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:14.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:13 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:13.719+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:14.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:13 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:13.719+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:14.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:13 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:13.719+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:14.146 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:13 smithi078 podman[84540]: 2024-04-25 01:49:13.941299305 +0000 UTC m=+5.619873044 container died 160409d9129f63f88dffbad352929a25b5ce4c038aeb3643ef8ceb3e4e4c9f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T01:49:14.147 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:13 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:13.890+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:14.147 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:13 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:13.890+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:14.147 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:13 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:13.890+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:14.147 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:13 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:13.890+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:14.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:14 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:14.100+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:14.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:14 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:14.100+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:14.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:14 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:14.100+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.586461+0000 front 2024-04-25T01:48:48.586480+0000 (oldest deadline 2024-04-25T01:49:12.686307+0000) 2024-04-25T01:49:14.664 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:14 smithi078 podman[84540]: 2024-04-25 01:49:14.44097931 +0000 UTC m=+6.119553054 container cleanup 160409d9129f63f88dffbad352929a25b5ce4c038aeb3643ef8ceb3e4e4c9f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T01:49:14.665 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:14 smithi078 bash[84540]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6 2024-04-25T01:49:14.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:14 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:14.380+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:14.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:14 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:14.380+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:14.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:14 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:14.380+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:14.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:14 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:14.380+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:14.979 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:14 smithi078 podman[84559]: 2024-04-25 01:49:14.668383784 +0000 UTC m=+0.725260856 container remove 160409d9129f63f88dffbad352929a25b5ce4c038aeb3643ef8ceb3e4e4c9f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T01:49:14.980 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:14 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:14.906+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:14.980 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:14 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:14.906+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:14.980 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:14 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:14.906+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:14.980 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:14 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:14.906+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:15.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:14 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:14.819+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:15.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:14 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:14.819+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:15.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:14 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:14.819+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:15.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:14 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:14.712+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:15.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:14 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:14.712+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:15.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:14 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:14.712+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:15.262 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:14 smithi078 podman[84661]: 2024-04-25 01:49:14.886132482 +0000 UTC m=+0.018054552 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:49:15.263 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:15 smithi078 podman[84661]: 2024-04-25 01:49:15.027628799 +0000 UTC m=+0.159550841 container create c1118139f06eb65921d056a1079beb81752d1c8c154463a2cb418653cd960f73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-04-25T01:49:15.407 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:15 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:15.075+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:15.407 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:15 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:15.075+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:15.407 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:15 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:15.075+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.586461+0000 front 2024-04-25T01:48:48.586480+0000 (oldest deadline 2024-04-25T01:49:12.686307+0000) 2024-04-25T01:49:15.584 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:15 smithi078 podman[84661]: 2024-04-25 01:49:15.281522834 +0000 UTC m=+0.413444885 container init c1118139f06eb65921d056a1079beb81752d1c8c154463a2cb418653cd960f73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T01:49:15.584 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:15 smithi078 podman[84661]: 2024-04-25 01:49:15.285264172 +0000 UTC m=+0.417186206 container start c1118139f06eb65921d056a1079beb81752d1c8c154463a2cb418653cd960f73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T01:49:15.584 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:15 smithi078 podman[84661]: 2024-04-25 01:49:15.380423941 +0000 UTC m=+0.512345992 container attach c1118139f06eb65921d056a1079beb81752d1c8c154463a2cb418653cd960f73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:49:15.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:15 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:15.409+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:15.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:15 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:15.409+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:15.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:15 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:15.409+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:15.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:15 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:15.409+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:15.859 INFO:journalctl@ceph.osd.6.smithi078.stdout:Apr 25 01:49:15 smithi078 podman[84661]: 2024-04-25 01:49:15.588331943 +0000 UTC m=+0.720254000 container died c1118139f06eb65921d056a1079beb81752d1c8c154463a2cb418653cd960f73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-25T01:49:16.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:15 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:15.740+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:16.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:15 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:15.740+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:16.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:15 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:15.740+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:16.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:15 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:15.783+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:16.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:15 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:15.783+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:16.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:15 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:15.783+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:16.248 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:15 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:15.862+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:16.248 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:15 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:15.862+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:16.248 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:15 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:15.862+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:16.248 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:15 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:15.862+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:16.275 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T01:49:16.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:16 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:16.044+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:16.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:16 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:16.044+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:16.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:16 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:16.044+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.586461+0000 front 2024-04-25T01:48:48.586480+0000 (oldest deadline 2024-04-25T01:49:12.686307+0000) 2024-04-25T01:49:16.360 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:49:16.361 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T01:49:16.361 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T01:49:16.361 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.7 2024-04-25T01:49:16.804 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:16 smithi152 systemd[1]: Stopping Ceph osd.7 for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:49:16.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:16 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:16.420+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:16.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:16 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:16.420+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:16.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:16 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:16.420+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:16.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:16 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:16.420+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:17.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:16 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:16.784+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:17.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:16 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:16.784+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:17.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:16 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:16.784+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:17.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:16 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:16.766+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:17.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:16 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:16.766+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:17.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:16 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:16.766+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:17.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:16 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:16.874+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:17.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:16 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:16.874+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:17.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:16 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:16.874+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:17.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:16 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:16.874+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:17.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:16 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:16.881+0000 7fd409632700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:49:17.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:16 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:16.881+0000 7fd409632700 -1 osd.7 98 *** Got signal Terminated *** 2024-04-25T01:49:17.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:16 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:16.881+0000 7fd409632700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:49:17.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:17 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:17.053+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:17.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:17 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:17.053+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:17.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:17 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:17.053+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.586461+0000 front 2024-04-25T01:48:48.586480+0000 (oldest deadline 2024-04-25T01:49:12.686307+0000) 2024-04-25T01:49:17.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:17 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:17.373+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:17.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:17 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:17.373+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:17.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:17 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:17.374+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:17.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:17 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:17.374+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:18.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:17 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:17.751+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:18.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:17 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:17.751+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:18.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:17 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:17.751+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:18.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:17 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:17.822+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:18.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:17 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:17.822+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:18.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:17 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:17.822+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:18.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:17 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:17.911+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:18.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:17 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:17.911+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:18.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:17 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:17.911+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:18.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:17 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:17.911+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:18.394 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:18 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:18.066+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:18.394 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:18 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:18.066+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:18.394 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:18 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:18.066+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.586461+0000 front 2024-04-25T01:48:48.586480+0000 (oldest deadline 2024-04-25T01:49:12.686307+0000) 2024-04-25T01:49:18.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:18 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:18.397+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:18.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:18 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:18.397+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:18.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:18 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:18.397+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:18.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:18 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:18.397+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:18.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:18 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:18.397+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:19.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:18 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:18.798+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:19.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:18 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:18.798+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:19.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:18 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:18.798+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:19.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:18 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:18.795+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:19.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:18 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:18.795+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:19.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:18 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:18.795+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:19.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:18 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:18.907+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:19.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:18 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:18.907+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:19.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:18 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:18.907+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:19.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:18 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:18.907+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:19.439 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:19 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:19.066+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:19.439 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:19 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:19.066+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:19.439 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:19 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:19.066+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.586461+0000 front 2024-04-25T01:48:48.586480+0000 (oldest deadline 2024-04-25T01:49:12.686307+0000) 2024-04-25T01:49:19.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:19 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:19.441+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:19.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:19 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:19.441+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:19.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:19 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:19.441+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:19.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:19 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:19.441+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:19.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:19 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:19.441+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:20.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:19 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:19.829+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:20.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:19 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:19.829+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:20.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:19 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:19.829+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:20.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:19 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:19.829+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:55.475699+0000 front 2024-04-25T01:48:55.475795+0000 (oldest deadline 2024-04-25T01:49:18.975788+0000) 2024-04-25T01:49:20.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:19 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:19.783+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:20.125 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:19 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:19.783+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:20.125 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:19 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:19.783+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:20.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:19 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:19.935+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:20.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:19 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:19.935+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:20.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:19 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:19.935+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:20.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:19 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:19.935+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:20.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:19 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:19.935+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:20.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:20 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:20.039+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:20.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:20 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:20.039+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:20.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:20 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:20.039+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.586461+0000 front 2024-04-25T01:48:48.586480+0000 (oldest deadline 2024-04-25T01:49:12.686307+0000) 2024-04-25T01:49:20.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:20 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:20.039+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.187250+0000 front 2024-04-25T01:48:53.187231+0000 (oldest deadline 2024-04-25T01:49:19.087371+0000) 2024-04-25T01:49:20.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:20 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:20.426+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:20.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:20 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:20.426+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:20.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:20 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:20.426+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:20.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:20 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:20.426+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:20.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:20 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:20.426+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:21.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:20 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:20.833+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:21.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:20 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:20.833+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:21.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:20 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:20.833+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:21.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:20 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:20.833+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:55.475699+0000 front 2024-04-25T01:48:55.475795+0000 (oldest deadline 2024-04-25T01:49:18.975788+0000) 2024-04-25T01:49:21.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:20 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:20.819+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:21.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:20 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:20.819+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:21.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:20 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:20.819+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:21.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:20 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:20.819+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:21.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:20 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:20.985+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:21.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:20 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:20.985+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:21.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:20 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:20.985+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:21.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:20 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:20.985+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:21.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:20 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:20.985+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:21.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:21 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:20.999+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:30.582878+0000 front 2024-04-25T01:48:30.582857+0000 (oldest deadline 2024-04-25T01:48:55.882985+0000) 2024-04-25T01:49:21.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:21 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:20.999+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.084634+0000 front 2024-04-25T01:48:38.084656+0000 (oldest deadline 2024-04-25T01:49:02.184729+0000) 2024-04-25T01:49:21.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:21 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:20.999+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.586461+0000 front 2024-04-25T01:48:48.586480+0000 (oldest deadline 2024-04-25T01:49:12.686307+0000) 2024-04-25T01:49:21.304 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:21 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7[50526]: 2024-04-25T01:49:20.999+0000 7fd405c26700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.187250+0000 front 2024-04-25T01:48:53.187231+0000 (oldest deadline 2024-04-25T01:49:19.087371+0000) 2024-04-25T01:49:21.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:21 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:21.399+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:21.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:21 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:21.399+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:21.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:21 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:21.399+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:21.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:21 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:21.399+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:21.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:21 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:21.399+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:22.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:21 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:21.833+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:22.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:21 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:21.833+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:22.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:21 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:21.833+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:22.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:21 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:21.833+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:55.475699+0000 front 2024-04-25T01:48:55.475795+0000 (oldest deadline 2024-04-25T01:49:18.975788+0000) 2024-04-25T01:49:22.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:21 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:21.770+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:22.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:21 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:21.770+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:22.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:21 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:21.770+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:22.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:21 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:21.770+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:22.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:22 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:22.008+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:22.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:22 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:22.008+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:22.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:22 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:22.008+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:22.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:22 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:22.008+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:22.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:22 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:22.008+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:22.552 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:22 smithi152 podman[66419]: 2024-04-25 01:49:22.253437243 +0000 UTC m=+5.698899869 container died cd4a667653b0b53f4425ad504e8cd0e9912b3aac418265de4183e08434f2e653 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T01:49:22.552 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:22 smithi152 podman[66419]: 2024-04-25 01:49:22.555704091 +0000 UTC m=+6.001166766 container cleanup cd4a667653b0b53f4425ad504e8cd0e9912b3aac418265de4183e08434f2e653 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T01:49:22.552 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:22 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:22.421+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:22.552 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:22 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:22.421+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:22.552 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:22 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:22.421+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:22.553 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:22 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:22.421+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:22.553 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:22 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:22.421+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:22.804 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:22 smithi152 bash[66419]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7 2024-04-25T01:49:23.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:22 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:22.828+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:23.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:22 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:22.828+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:23.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:22 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:22.828+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:23.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:22 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:22.828+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:55.475699+0000 front 2024-04-25T01:48:55.475795+0000 (oldest deadline 2024-04-25T01:49:18.975788+0000) 2024-04-25T01:49:23.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:22 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:22.791+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:23.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:22 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:22.791+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:23.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:22 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:22.791+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:23.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:22 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:22.791+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:23.163 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:22 smithi152 podman[66437]: 2024-04-25 01:49:22.832463049 +0000 UTC m=+0.577408785 container remove cd4a667653b0b53f4425ad504e8cd0e9912b3aac418265de4183e08434f2e653 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-25T01:49:23.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:22 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:22.970+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:23.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:22 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:22.970+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:23.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:22 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:22.970+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:23.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:22 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:22.970+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:23.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:22 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:22.970+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:23.433 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:23 smithi152 podman[66530]: 2024-04-25 01:49:23.166644722 +0000 UTC m=+0.113833873 container create 8c0027f1e8f4c95ab3330e2b885cac837e32fe3552bde7570e32c7d85d7fdb72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:49:23.433 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:23 smithi152 podman[66530]: 2024-04-25 01:49:23.070761199 +0000 UTC m=+0.017950312 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:49:23.433 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:23 smithi152 podman[66530]: 2024-04-25 01:49:23.386629938 +0000 UTC m=+0.333819048 container init 8c0027f1e8f4c95ab3330e2b885cac837e32fe3552bde7570e32c7d85d7fdb72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T01:49:23.433 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:23 smithi152 podman[66530]: 2024-04-25 01:49:23.389741822 +0000 UTC m=+0.336930932 container start 8c0027f1e8f4c95ab3330e2b885cac837e32fe3552bde7570e32c7d85d7fdb72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T01:49:23.804 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:23 smithi152 podman[66530]: 2024-04-25 01:49:23.480297479 +0000 UTC m=+0.427486604 container attach 8c0027f1e8f4c95ab3330e2b885cac837e32fe3552bde7570e32c7d85d7fdb72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T01:49:23.805 INFO:journalctl@ceph.osd.7.smithi152.stdout:Apr 25 01:49:23 smithi152 podman[66530]: 2024-04-25 01:49:23.684094879 +0000 UTC m=+0.631283992 container died 8c0027f1e8f4c95ab3330e2b885cac837e32fe3552bde7570e32c7d85d7fdb72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T01:49:23.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:23 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:23.435+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:23.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:23 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:23.435+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:23.806 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:23 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:23.435+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:23.806 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:23 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:23.435+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:23.806 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:23 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:23.435+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:24.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:23 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:23.852+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:24.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:23 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:23.852+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:24.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:23 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:23.852+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:24.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:23 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:23.852+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:55.475699+0000 front 2024-04-25T01:48:55.475795+0000 (oldest deadline 2024-04-25T01:49:18.975788+0000) 2024-04-25T01:49:24.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:23 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:23.764+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:24.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:23 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:23.764+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:24.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:23 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:23.764+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:24.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:23 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:23.764+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:24.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:23 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:23.970+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:24.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:23 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:23.970+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:24.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:23 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:23.970+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:24.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:23 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:23.970+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:24.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:23 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:23.970+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:24.310 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T01:49:24.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:49:24.349 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T01:49:24.350 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T01:49:24.350 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.8 2024-04-25T01:49:24.554 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:24 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:24.461+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:24.554 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:24 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:24.461+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:24.555 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:24 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:24.461+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:24.555 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:24 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:24.461+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:24.555 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:24 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:24.461+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:24.644 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:24 smithi196 systemd[1]: Stopping Ceph osd.8 for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:49:25.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:24 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:24.720+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:25.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:24 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:24.720+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:25.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:24 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:24.720+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:25.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:24 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:24.720+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:25.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:24 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:24.646+0000 7f855475d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:49:25.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:24 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:24.646+0000 7f855475d700 -1 osd.8 98 *** Got signal Terminated *** 2024-04-25T01:49:25.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:24 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:24.646+0000 7f855475d700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:49:25.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:24 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:24.884+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:25.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:24 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:24.884+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:25.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:24 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:24.884+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:25.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:24 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:24.884+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:55.475699+0000 front 2024-04-25T01:48:55.475795+0000 (oldest deadline 2024-04-25T01:49:18.975788+0000) 2024-04-25T01:49:25.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:24 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:24.934+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:25.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:24 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:24.934+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:25.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:24 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:24.934+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:25.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:24 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:24.934+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:25.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:24 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:24.934+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:25.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:25 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:25.461+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:25.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:25 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:25.461+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:25.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:25 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:25.461+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:25.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:25 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:25.461+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:25.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:25 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:25.461+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:26.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:25 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:25.933+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:26.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:25 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:25.933+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:26.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:25 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:25.933+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:26.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:25 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:25.933+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:55.475699+0000 front 2024-04-25T01:48:55.475795+0000 (oldest deadline 2024-04-25T01:49:18.975788+0000) 2024-04-25T01:49:26.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:25 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:25.745+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:26.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:25 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:25.745+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:26.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:25 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:25.745+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:26.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:25 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:25.745+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:26.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:25 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:25.888+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:26.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:25 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:25.888+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:26.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:25 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:25.888+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:26.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:25 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:25.888+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:26.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:25 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:25.888+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:26.500 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:26 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:26.500+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:26.500 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:26 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:26.500+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:26.500 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:26 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:26.500+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:26.500 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:26 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:26.500+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:26.500 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:26 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:26.500+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:26.500 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:26 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:26.500+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:26.971 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:26.715+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:26.971 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:26.715+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:26.971 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:26.715+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:26.971 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:26.715+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:27.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:26 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:26.937+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:27.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:26 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:26.937+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:27.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:26 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:26.937+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:27.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:26 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:26.937+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:27.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:26 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:26.937+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:27.372 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:26.974+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:27.372 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:26.974+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:27.372 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:26.974+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:27.372 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:26.974+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:55.475699+0000 front 2024-04-25T01:48:55.475795+0000 (oldest deadline 2024-04-25T01:49:18.975788+0000) 2024-04-25T01:49:27.372 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:26 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:26.974+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:02.476708+0000 front 2024-04-25T01:49:02.476817+0000 (oldest deadline 2024-04-25T01:49:25.976819+0000) 2024-04-25T01:49:27.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:27 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:27.507+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:27.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:27 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:27.507+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:27.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:27 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:27.507+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:27.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:27 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:27.507+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:27.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:27 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:27.507+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:27.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:27 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:27.507+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:28.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:27 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:27.744+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:28.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:27 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:27.744+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:28.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:27 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:27.744+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:28.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:27 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:27.744+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:28.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:27 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:27.744+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:28.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:27 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:27.952+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:28.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:27 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:27.952+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:28.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:27 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:27.952+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:28.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:27 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:27.952+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:55.475699+0000 front 2024-04-25T01:48:55.475795+0000 (oldest deadline 2024-04-25T01:49:18.975788+0000) 2024-04-25T01:49:28.123 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:27 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:27.952+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:02.476708+0000 front 2024-04-25T01:49:02.476817+0000 (oldest deadline 2024-04-25T01:49:25.976819+0000) 2024-04-25T01:49:28.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:27 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:27.945+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:28.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:27 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:27.945+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:28.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:27 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:27.945+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:28.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:27 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:27.945+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:28.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:27 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:27.945+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:28.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:28 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:28.463+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:28.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:28 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:28.463+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:28.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:28 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:28.463+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:28.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:28 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:28.463+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:28.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:28 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:28.463+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:28.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:28 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:28.463+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:29.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:28.907+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:28.571686+0000 front 2024-04-25T01:48:28.571726+0000 (oldest deadline 2024-04-25T01:48:53.871683+0000) 2024-04-25T01:49:29.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:28.907+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:39.172592+0000 front 2024-04-25T01:48:39.172709+0000 (oldest deadline 2024-04-25T01:49:01.472853+0000) 2024-04-25T01:49:29.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:28.907+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.474360+0000 front 2024-04-25T01:48:48.474344+0000 (oldest deadline 2024-04-25T01:49:12.574390+0000) 2024-04-25T01:49:29.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:28.907+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:55.475699+0000 front 2024-04-25T01:48:55.475795+0000 (oldest deadline 2024-04-25T01:49:18.975788+0000) 2024-04-25T01:49:29.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8[52068]: 2024-04-25T01:49:28.907+0000 7f854cd49700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:02.476708+0000 front 2024-04-25T01:49:02.476817+0000 (oldest deadline 2024-04-25T01:49:25.976819+0000) 2024-04-25T01:49:29.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:28.748+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:29.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:28.748+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:29.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:28.748+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:29.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:28.748+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:29.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:28.748+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:29.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:28 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:28.937+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:29.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:28 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:28.937+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:29.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:28 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:28.937+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:29.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:28 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:28.937+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:29.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:28 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:28.937+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:29.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:29 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:29.473+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:29.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:29 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:29.473+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:29.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:29 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:29.473+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:29.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:29 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:29.473+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:29.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:29 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:29.474+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:29.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:29 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:29.474+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:30.116 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:29 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:29.763+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:30.116 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:29 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:29.763+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:30.116 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:29 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:29.763+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:30.116 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:29 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:29.763+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:30.116 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:29 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:29.763+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:30.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:29 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:29.935+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:30.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:29 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:29.935+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:30.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:29 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:29.935+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:30.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:29 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:29.935+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:30.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:29 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:29.935+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:30.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:29 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:29.935+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.310234+0000 front 2024-04-25T01:49:04.310148+0000 (oldest deadline 2024-04-25T01:49:29.609949+0000) 2024-04-25T01:49:30.372 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:30 smithi196 podman[68019]: 2024-04-25 01:49:30.11847263 +0000 UTC m=+5.570520508 container died dffdae8b1b29d4ca58299473dfe0c7cc82e1f3d71cc5699c4cde49d5c40062d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T01:49:30.712 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:30 smithi196 podman[68019]: 2024-04-25 01:49:30.51476415 +0000 UTC m=+5.966812029 container cleanup dffdae8b1b29d4ca58299473dfe0c7cc82e1f3d71cc5699c4cde49d5c40062d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T01:49:30.713 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:30 smithi196 bash[68019]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8 2024-04-25T01:49:30.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:30 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:30.496+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:30.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:30 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:30.496+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:30.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:30 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:30.496+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:30.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:30 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:30.496+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:30.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:30 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:30.496+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:30.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:30 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:30.496+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:31.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:30 smithi196 podman[68037]: 2024-04-25 01:49:30.715980052 +0000 UTC m=+0.596182582 container remove dffdae8b1b29d4ca58299473dfe0c7cc82e1f3d71cc5699c4cde49d5c40062d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T01:49:31.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:30 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:30.721+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:31.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:30 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:30.721+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:31.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:30 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:30.721+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:31.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:30 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:30.721+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:31.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:30 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:30.721+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:31.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:30 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:30.927+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:31.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:30 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:30.927+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:31.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:30 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:30.927+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:31.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:30 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:30.927+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:31.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:30 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:30.927+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:31.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:30 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:30.927+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.310234+0000 front 2024-04-25T01:49:04.310148+0000 (oldest deadline 2024-04-25T01:49:29.609949+0000) 2024-04-25T01:49:31.483 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:31 smithi196 podman[68129]: 2024-04-25 01:49:31.034339485 +0000 UTC m=+0.018302776 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:49:31.483 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:31 smithi196 podman[68129]: 2024-04-25 01:49:31.208689921 +0000 UTC m=+0.192653242 container create a6b5ae3b0eea69b461c18f00cc6d63326d80752b92b6b332535d6820cb0e4658 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T01:49:31.483 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:31 smithi196 podman[68129]: 2024-04-25 01:49:31.486784166 +0000 UTC m=+0.470747447 container init a6b5ae3b0eea69b461c18f00cc6d63326d80752b92b6b332535d6820cb0e4658 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-25T01:49:31.774 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:31 smithi196 podman[68129]: 2024-04-25 01:49:31.49037231 +0000 UTC m=+0.474335599 container start a6b5ae3b0eea69b461c18f00cc6d63326d80752b92b6b332535d6820cb0e4658 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T01:49:31.774 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:31 smithi196 podman[68129]: 2024-04-25 01:49:31.586571688 +0000 UTC m=+0.570534969 container attach a6b5ae3b0eea69b461c18f00cc6d63326d80752b92b6b332535d6820cb0e4658 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:49:31.775 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:31 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:31.676+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:31.775 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:31 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:31.676+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:31.775 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:31 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:31.676+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:31.775 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:31 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:31.676+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:31.775 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:31 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:31.676+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:31.806 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:31 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:31.541+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:31.807 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:31 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:31.541+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:31.807 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:31 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:31.541+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:31.807 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:31 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:31.541+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:31.807 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:31 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:31.541+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:31.807 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:31 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:31.541+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:32.122 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 25 01:49:31 smithi196 podman[68129]: 2024-04-25 01:49:31.777907853 +0000 UTC m=+0.761871132 container died a6b5ae3b0eea69b461c18f00cc6d63326d80752b92b6b332535d6820cb0e4658 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-25T01:49:32.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:31 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:31.900+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:32.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:31 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:31.900+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:32.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:31 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:31.900+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:32.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:31 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:31.900+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:32.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:31 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:31.900+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:32.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:31 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:31.900+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.310234+0000 front 2024-04-25T01:49:04.310148+0000 (oldest deadline 2024-04-25T01:49:29.609949+0000) 2024-04-25T01:49:32.461 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T01:49:32.499 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:49:32.499 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T01:49:32.500 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T01:49:32.500 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.9 2024-04-25T01:49:32.784 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:32 smithi078 systemd[1]: Stopping Ceph osd.9 for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:49:32.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:32 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:32.518+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:32.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:32 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:32.518+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:32.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:32 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:32.518+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:32.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:32 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:32.518+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:32.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:32 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:32.518+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:32.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:32 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:32.518+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:33.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:32 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:32.712+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:33.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:32 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:32.712+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:33.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:32 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:32.712+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:33.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:32 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:32.712+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:33.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:32 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:32.712+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:33.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:32 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:32.786+0000 7f20dca49700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:49:33.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:32 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:32.786+0000 7f20dca49700 -1 osd.9 98 *** Got signal Terminated *** 2024-04-25T01:49:33.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:32 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:32.786+0000 7f20dca49700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:49:33.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:32 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:32.897+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:33.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:32 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:32.897+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:33.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:32 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:32.897+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:33.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:32 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:32.897+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:33.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:32 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:32.897+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:33.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:32 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:32.897+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.310234+0000 front 2024-04-25T01:49:04.310148+0000 (oldest deadline 2024-04-25T01:49:29.609949+0000) 2024-04-25T01:49:33.729 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:33 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:33.475+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:33.729 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:33 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:33.475+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:33.729 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:33 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:33.475+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:33.729 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:33 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:33.475+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:33.729 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:33 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:33.475+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:33.729 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:33 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:33.475+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:34.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:33 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:33.704+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:34.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:33 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:33.704+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:34.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:33 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:33.704+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:34.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:33 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:33.704+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:34.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:33 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:33.704+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:34.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:33 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:33.939+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:34.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:33 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:33.939+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:34.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:33 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:33.939+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:34.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:33 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:33.939+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:34.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:33 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:33.939+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:34.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:33 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:33.939+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.310234+0000 front 2024-04-25T01:49:04.310148+0000 (oldest deadline 2024-04-25T01:49:29.609949+0000) 2024-04-25T01:49:34.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:34 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:34.481+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:34.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:34 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:34.481+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:34.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:34 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:34.481+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:34.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:34 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:34.481+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:34.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:34 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:34.481+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:34.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:34 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:34.481+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:35.023 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:34 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:34.661+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:35.024 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:34 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:34.661+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:35.024 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:34 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:34.661+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:35.024 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:34 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:34.661+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:35.024 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:34 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:34.661+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:35.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:34 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:34.965+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:35.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:34 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:34.965+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:35.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:34 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:34.965+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:35.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:34 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:34.965+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:35.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:34 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:34.965+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:35.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:34 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:34.965+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.310234+0000 front 2024-04-25T01:49:04.310148+0000 (oldest deadline 2024-04-25T01:49:29.609949+0000) 2024-04-25T01:49:35.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:35 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:35.480+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:35.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:35 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:35.480+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:35.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:35 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:35.480+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:35.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:35 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:35.480+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:35.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:35 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:35.480+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:35.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:35 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:35.480+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:36.035 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:35 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:35.664+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:36.035 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:35 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:35.664+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:36.035 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:35 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:35.664+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:36.035 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:35 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:35.664+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:36.036 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:35 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:35.664+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:36.036 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:35 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:35.664+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:36.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:35 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:35.983+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:36.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:35 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:35.983+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:36.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:35 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:35.983+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:36.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:35 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:35.983+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:36.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:35 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:35.983+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:36.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:35 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:35.983+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.310234+0000 front 2024-04-25T01:49:04.310148+0000 (oldest deadline 2024-04-25T01:49:29.609949+0000) 2024-04-25T01:49:36.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:36 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:36.445+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:36.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:36 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:36.445+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:36.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:36 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:36.445+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:36.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:36 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:36.445+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:36.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:36 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:36.445+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:36.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:36 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:36.445+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:37.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:36 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:36.699+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:37.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:36 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:36.699+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:37.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:36 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:36.699+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:37.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:36 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:36.699+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:37.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:36 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:36.699+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:37.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:36 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:36.699+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:37.262 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:36 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:36.979+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.703775+0000 front 2024-04-25T01:48:21.703758+0000 (oldest deadline 2024-04-25T01:48:47.003928+0000) 2024-04-25T01:49:37.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:36 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:36.979+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:27.004570+0000 front 2024-04-25T01:48:27.004529+0000 (oldest deadline 2024-04-25T01:48:58.805061+0000) 2024-04-25T01:49:37.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:36 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:36.979+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.805572+0000 front 2024-04-25T01:48:38.805594+0000 (oldest deadline 2024-04-25T01:49:02.305476+0000) 2024-04-25T01:49:37.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:36 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:36.979+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:48.107110+0000 front 2024-04-25T01:48:48.107123+0000 (oldest deadline 2024-04-25T01:49:09.807300+0000) 2024-04-25T01:49:37.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:36 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:36.979+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:56.208779+0000 front 2024-04-25T01:48:56.208751+0000 (oldest deadline 2024-04-25T01:49:19.108730+0000) 2024-04-25T01:49:37.263 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:36 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9[62059]: 2024-04-25T01:49:36.979+0000 7f20d5035700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.310234+0000 front 2024-04-25T01:49:04.310148+0000 (oldest deadline 2024-04-25T01:49:29.609949+0000) 2024-04-25T01:49:37.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:37 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:37.490+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:37.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:37 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:37.490+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:37.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:37 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:37.490+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:37.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:37 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:37.490+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:37.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:37 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:37.490+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:37.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:37 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:37.490+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:38.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:37 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:37.657+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:38.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:37 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:37.657+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:38.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:37 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:37.657+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:38.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:37 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:37.657+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:38.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:37 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:37.657+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:38.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:37 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:37.657+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:38.376 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:38 smithi078 podman[84769]: 2024-04-25 01:49:38.127882875 +0000 UTC m=+5.436843608 container died 0ef4fe01cdee8a7449b531fba90c0ba959d13be3013218a9246c5831162950ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:49:38.763 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:38 smithi078 podman[84769]: 2024-04-25 01:49:38.562328762 +0000 UTC m=+5.871289515 container cleanup 0ef4fe01cdee8a7449b531fba90c0ba959d13be3013218a9246c5831162950ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-25T01:49:38.763 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:38 smithi078 bash[84769]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9 2024-04-25T01:49:38.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:38 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:38.448+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:38.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:38 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:38.448+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:38.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:38 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:38.448+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:38.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:38 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:38.448+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:38.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:38 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:38.448+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:38.806 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:38 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:38.448+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:38.806 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:38 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:38.448+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:12.122299+0000 front 2024-04-25T01:49:12.122360+0000 (oldest deadline 2024-04-25T01:49:38.022169+0000) 2024-04-25T01:49:39.082 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:38 smithi078 podman[84786]: 2024-04-25 01:49:38.797157531 +0000 UTC m=+0.662348886 container remove 0ef4fe01cdee8a7449b531fba90c0ba959d13be3013218a9246c5831162950ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T01:49:39.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:38 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:38.705+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:39.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:38 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:38.705+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:39.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:38 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:38.705+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:39.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:38 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:38.705+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:39.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:38 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:38.705+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:39.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:38 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:38.705+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:39.380 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:39 smithi078 podman[84876]: 2024-04-25 01:49:38.98900044 +0000 UTC m=+0.017816985 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:49:39.380 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:39 smithi078 podman[84876]: 2024-04-25 01:49:39.089584258 +0000 UTC m=+0.118400788 container create 57a5ccb422708d1248bee91d1caf8c5e60df619349813d293bb1cc9d655a8087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T01:49:39.380 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:39 smithi078 podman[84876]: 2024-04-25 01:49:39.300539755 +0000 UTC m=+0.329356289 container init 57a5ccb422708d1248bee91d1caf8c5e60df619349813d293bb1cc9d655a8087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T01:49:39.381 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:39 smithi078 podman[84876]: 2024-04-25 01:49:39.303545624 +0000 UTC m=+0.332362154 container start 57a5ccb422708d1248bee91d1caf8c5e60df619349813d293bb1cc9d655a8087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-25T01:49:39.381 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:39 smithi078 podman[84876]: 2024-04-25 01:49:39.383641038 +0000 UTC m=+0.412457584 container attach 57a5ccb422708d1248bee91d1caf8c5e60df619349813d293bb1cc9d655a8087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:49:39.763 INFO:journalctl@ceph.osd.9.smithi078.stdout:Apr 25 01:49:39 smithi078 podman[84876]: 2024-04-25 01:49:39.59400502 +0000 UTC m=+0.622821542 container died 57a5ccb422708d1248bee91d1caf8c5e60df619349813d293bb1cc9d655a8087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T01:49:39.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:39 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:39.418+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:39.806 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:39 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:39.418+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:39.806 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:39 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:39.418+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:39.806 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:39 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:39.418+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:39.806 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:39 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:39.418+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:39.806 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:39 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:39.418+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:39.806 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:39 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:39.418+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:12.122299+0000 front 2024-04-25T01:49:12.122360+0000 (oldest deadline 2024-04-25T01:49:38.022169+0000) 2024-04-25T01:49:40.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:39 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:39.671+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:40.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:39 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:39.671+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:40.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:39 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:39.671+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:40.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:39 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:39.671+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:40.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:39 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:39.671+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:40.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:39 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:39.671+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:40.261 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T01:49:40.298 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:49:40.299 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T01:49:40.299 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T01:49:40.299 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.10 2024-04-25T01:49:40.614 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:40 smithi152 systemd[1]: Stopping Ceph osd.10 for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:49:40.614 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:40 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:40.444+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:40.614 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:40 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:40.444+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:40.615 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:40 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:40.444+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:40.615 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:40 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:40.444+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:40.615 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:40 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:40.444+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:40.615 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:40 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:40.444+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:40.615 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:40 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:40.444+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:12.122299+0000 front 2024-04-25T01:49:12.122360+0000 (oldest deadline 2024-04-25T01:49:38.022169+0000) 2024-04-25T01:49:40.615 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:40 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:40.616+0000 7f422870b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:49:40.615 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:40 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:40.616+0000 7f422870b700 -1 osd.10 98 *** Got signal Terminated *** 2024-04-25T01:49:40.615 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:40 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:40.616+0000 7f422870b700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:49:41.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:40 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:40.702+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:41.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:40 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:40.702+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:41.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:40 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:40.702+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:41.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:40 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:40.702+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:41.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:40 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:40.702+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:41.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:40 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:40.702+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:41.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:41 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:41.418+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:41.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:41 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:41.418+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:41.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:41 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:41.418+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:41.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:41 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:41.418+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:41.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:41 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:41.418+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:41.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:41 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:41.418+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:41.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:41 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:41.418+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:12.122299+0000 front 2024-04-25T01:49:12.122360+0000 (oldest deadline 2024-04-25T01:49:38.022169+0000) 2024-04-25T01:49:42.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:41 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:41.744+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:42.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:41 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:41.744+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:42.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:41 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:41.744+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:42.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:41 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:41.744+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:42.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:41 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:41.744+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:42.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:41 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:41.744+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:42.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:42 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:42.422+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:42.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:42 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:42.422+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:42.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:42 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:42.422+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:42.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:42 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:42.422+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:42.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:42 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:42.422+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:42.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:42 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:42.422+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:42.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:42 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:42.422+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:12.122299+0000 front 2024-04-25T01:49:12.122360+0000 (oldest deadline 2024-04-25T01:49:38.022169+0000) 2024-04-25T01:49:43.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:42 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:42.782+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:43.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:42 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:42.782+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:43.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:42 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:42.782+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:43.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:42 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:42.782+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:43.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:42 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:42.782+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:43.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:42 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:42.782+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:43.763 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:43 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:43.428+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:43.763 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:43 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:43.428+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:43.764 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:43 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:43.428+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:43.764 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:43 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:43.428+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:43.764 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:43 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:43.428+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:43.764 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:43 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:43.428+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:43.764 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:43 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:43.428+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:12.122299+0000 front 2024-04-25T01:49:12.122360+0000 (oldest deadline 2024-04-25T01:49:38.022169+0000) 2024-04-25T01:49:44.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:43 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:43.778+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:44.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:43 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:43.778+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:44.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:43 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:43.778+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:44.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:43 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:43.778+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:44.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:43 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:43.778+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:44.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:43 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:43.778+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:44.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:44 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:44.419+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:44.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:44 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:44.419+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:44.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:44 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:44.419+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:44.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:44 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:44.419+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:44.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:44 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:44.419+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:44.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:44 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:44.419+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:44.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:44 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:44.419+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:12.122299+0000 front 2024-04-25T01:49:12.122360+0000 (oldest deadline 2024-04-25T01:49:38.022169+0000) 2024-04-25T01:49:45.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:44 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:44.808+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:45.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:44 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:44.808+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:45.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:44 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:44.808+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:45.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:44 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:44.808+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:45.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:44 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:44.808+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:45.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:44 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:44.808+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:45.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:44 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:44.808+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2024-04-25T01:49:20.146697+0000 front 2024-04-25T01:49:20.146718+0000 (oldest deadline 2024-04-25T01:49:44.246581+0000) 2024-04-25T01:49:45.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:45 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:45.435+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:21.212263+0000 front 2024-04-25T01:48:21.212281+0000 (oldest deadline 2024-04-25T01:48:45.312204+0000) 2024-04-25T01:49:45.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:45 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:45.435+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:32.214604+0000 front 2024-04-25T01:48:32.214639+0000 (oldest deadline 2024-04-25T01:48:57.514689+0000) 2024-04-25T01:49:45.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:45 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:45.435+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:38.516209+0000 front 2024-04-25T01:48:38.516225+0000 (oldest deadline 2024-04-25T01:49:04.416340+0000) 2024-04-25T01:49:45.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:45 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:45.435+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-04-25T01:48:47.917267+0000 front 2024-04-25T01:48:47.917320+0000 (oldest deadline 2024-04-25T01:49:10.717996+0000) 2024-04-25T01:49:45.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:45 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:45.435+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.019026+0000 front 2024-04-25T01:48:53.019005+0000 (oldest deadline 2024-04-25T01:49:18.319095+0000) 2024-04-25T01:49:45.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:45 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:45.435+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:04.621286+0000 front 2024-04-25T01:49:04.621163+0000 (oldest deadline 2024-04-25T01:49:26.321062+0000) 2024-04-25T01:49:45.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:45 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10[57566]: 2024-04-25T01:49:45.435+0000 7f4220cf7700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:12.122299+0000 front 2024-04-25T01:49:12.122360+0000 (oldest deadline 2024-04-25T01:49:38.022169+0000) 2024-04-25T01:49:46.109 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:45 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:45.854+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:46.109 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:45 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:45.854+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:46.109 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:45 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:45.854+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:46.109 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:45 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:45.854+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:46.109 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:45 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:45.854+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:46.109 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:45 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:45.854+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:46.109 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:45 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:45.854+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2024-04-25T01:49:20.146697+0000 front 2024-04-25T01:49:20.146718+0000 (oldest deadline 2024-04-25T01:49:44.246581+0000) 2024-04-25T01:49:46.304 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:46 smithi152 podman[66648]: 2024-04-25 01:49:46.051103718 +0000 UTC m=+5.549683005 container died 123cf9b53c0a018d6135b170c7d719333b5f0d1edf67eead8ee75d0639274744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:49:46.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:46 smithi152 podman[66648]: 2024-04-25 01:49:46.39200028 +0000 UTC m=+5.890579560 container cleanup 123cf9b53c0a018d6135b170c7d719333b5f0d1edf67eead8ee75d0639274744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:49:46.804 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:46 smithi152 bash[66648]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10 2024-04-25T01:49:46.805 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:46 smithi152 podman[66665]: 2024-04-25 01:49:46.551428017 +0000 UTC m=+0.498977613 container remove 123cf9b53c0a018d6135b170c7d719333b5f0d1edf67eead8ee75d0639274744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:49:47.276 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:46 smithi152 podman[66754]: 2024-04-25 01:49:46.86735639 +0000 UTC m=+0.018119545 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:49:47.277 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:46 smithi152 podman[66754]: 2024-04-25 01:49:46.968891802 +0000 UTC m=+0.119654928 container create 25b4df542ac1a8be25fd92b81d30a09282a1e3c6037c0b0a8a94d292b75639fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T01:49:47.277 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:47 smithi152 podman[66754]: 2024-04-25 01:49:47.196647031 +0000 UTC m=+0.347410158 container init 25b4df542ac1a8be25fd92b81d30a09282a1e3c6037c0b0a8a94d292b75639fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T01:49:47.277 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:47 smithi152 podman[66754]: 2024-04-25 01:49:47.199780316 +0000 UTC m=+0.350543443 container start 25b4df542ac1a8be25fd92b81d30a09282a1e3c6037c0b0a8a94d292b75639fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T01:49:47.277 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:47 smithi152 podman[66754]: 2024-04-25 01:49:47.2797879 +0000 UTC m=+0.430551028 container attach 25b4df542ac1a8be25fd92b81d30a09282a1e3c6037c0b0a8a94d292b75639fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T01:49:47.373 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:46 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:46.894+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:47.373 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:46 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:46.894+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:47.374 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:46 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:46.894+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:47.374 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:46 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:46.894+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:47.374 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:46 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:46.894+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:47.374 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:46 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:46.894+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:47.374 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:46 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:46.894+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2024-04-25T01:49:20.146697+0000 front 2024-04-25T01:49:20.146718+0000 (oldest deadline 2024-04-25T01:49:44.246581+0000) 2024-04-25T01:49:47.554 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:47 smithi152 podman[66754]: 2024-04-25 01:49:47.492274429 +0000 UTC m=+0.643037575 container died 25b4df542ac1a8be25fd92b81d30a09282a1e3c6037c0b0a8a94d292b75639fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T01:49:48.003 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T01:49:48.026 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:47 smithi152 podman[66780]: 2024-04-25 01:49:47.991515229 +0000 UTC m=+0.494164533 container remove 25b4df542ac1a8be25fd92b81d30a09282a1e3c6037c0b0a8a94d292b75639fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0) 2024-04-25T01:49:48.026 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:48 smithi152 systemd[1]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-25T01:49:48.026 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:48 smithi152 systemd[1]: Stopped Ceph osd.10 for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:49:48.026 INFO:journalctl@ceph.osd.10.smithi152.stdout:Apr 25 01:49:48 smithi152 systemd[1]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.10.service: Consumed 10.965s CPU time. 2024-04-25T01:49:48.041 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:49:48.041 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T01:49:48.041 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T01:49:48.041 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.11 2024-04-25T01:49:48.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:47 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:47.868+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:48.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:47 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:47.868+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:48.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:47 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:47.868+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:48.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:47 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:47.868+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:48.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:47 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:47.868+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:48.124 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:47 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:47.868+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:48.124 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:47 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:47.868+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2024-04-25T01:49:20.146697+0000 front 2024-04-25T01:49:20.146718+0000 (oldest deadline 2024-04-25T01:49:44.246581+0000) 2024-04-25T01:49:48.124 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:48 smithi196 systemd[1]: Stopping Ceph osd.11 for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:49:48.622 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:48 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:48.367+0000 7fe12a240700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:49:48.623 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:48 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:48.367+0000 7fe12a240700 -1 osd.11 98 *** Got signal Terminated *** 2024-04-25T01:49:48.623 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:48 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:48.367+0000 7fe12a240700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:49:49.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:48 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:48.850+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:49.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:48 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:48.850+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:49.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:48 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:48.850+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:49.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:48 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:48.850+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:49.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:48 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:48.850+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:49.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:48 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:48.850+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:49.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:48 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:48.850+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2024-04-25T01:49:20.146697+0000 front 2024-04-25T01:49:20.146718+0000 (oldest deadline 2024-04-25T01:49:44.246581+0000) 2024-04-25T01:49:50.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:49 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:49.867+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:50.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:49 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:49.867+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:50.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:49 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:49.867+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:50.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:49 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:49.867+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:50.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:49 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:49.867+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:50.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:49 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:49.867+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:50.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:49 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:49.867+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2024-04-25T01:49:20.146697+0000 front 2024-04-25T01:49:20.146718+0000 (oldest deadline 2024-04-25T01:49:44.246581+0000) 2024-04-25T01:49:51.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:50 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:50.830+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:51.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:50 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:50.830+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:51.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:50 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:50.830+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:51.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:50 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:50.830+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:51.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:50 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:50.830+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:51.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:50 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:50.830+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:51.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:50 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:50.830+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2024-04-25T01:49:20.146697+0000 front 2024-04-25T01:49:20.146718+0000 (oldest deadline 2024-04-25T01:49:44.246581+0000) 2024-04-25T01:49:52.122 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:51 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:51.787+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:52.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:51 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:51.787+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:52.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:51 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:51.787+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:52.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:51 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:51.787+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:52.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:51 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:51.787+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:52.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:51 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:51.787+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:52.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:51 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:51.787+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2024-04-25T01:49:20.146697+0000 front 2024-04-25T01:49:20.146718+0000 (oldest deadline 2024-04-25T01:49:44.246581+0000) 2024-04-25T01:49:53.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:52 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:52.831+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-04-25T01:48:22.036072+0000 front 2024-04-25T01:48:22.036147+0000 (oldest deadline 2024-04-25T01:48:47.236738+0000) 2024-04-25T01:49:53.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:52 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:52.831+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6804 osd.1 since back 2024-04-25T01:48:31.937817+0000 front 2024-04-25T01:48:31.937662+0000 (oldest deadline 2024-04-25T01:48:57.237843+0000) 2024-04-25T01:49:53.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:52 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:52.831+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-25T01:48:37.738847+0000 front 2024-04-25T01:48:37.739018+0000 (oldest deadline 2024-04-25T01:49:01.239162+0000) 2024-04-25T01:49:53.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:52 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:52.831+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6812 osd.4 since back 2024-04-25T01:48:53.342880+0000 front 2024-04-25T01:48:53.342866+0000 (oldest deadline 2024-04-25T01:49:20.343514+0000) 2024-04-25T01:49:53.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:52 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:52.831+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-25T01:49:01.444450+0000 front 2024-04-25T01:49:01.444527+0000 (oldest deadline 2024-04-25T01:49:26.744483+0000) 2024-04-25T01:49:53.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:52 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:52.831+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-04-25T01:49:13.145812+0000 front 2024-04-25T01:49:13.145905+0000 (oldest deadline 2024-04-25T01:49:35.445752+0000) 2024-04-25T01:49:53.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:52 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:52.831+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.152:6820 osd.7 since back 2024-04-25T01:49:20.146697+0000 front 2024-04-25T01:49:20.146718+0000 (oldest deadline 2024-04-25T01:49:44.246581+0000) 2024-04-25T01:49:53.123 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:52 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11[58925]: 2024-04-25T01:49:52.831+0000 7fe126834700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-04-25T01:49:29.547311+0000 front 2024-04-25T01:49:29.547337+0000 (oldest deadline 2024-04-25T01:49:52.447396+0000) 2024-04-25T01:49:54.081 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:53 smithi196 podman[68246]: 2024-04-25 01:49:53.821483748 +0000 UTC m=+5.587185957 container died 7c46298081691f565ee2168ff505e1138998b760e8729cd244c09fb6d6c51894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T01:49:54.366 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:54 smithi196 podman[68246]: 2024-04-25 01:49:54.1586916 +0000 UTC m=+5.924393798 container cleanup 7c46298081691f565ee2168ff505e1138998b760e8729cd244c09fb6d6c51894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T01:49:54.366 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:54 smithi196 bash[68246]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11 2024-04-25T01:49:54.622 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:54 smithi196 podman[68263]: 2024-04-25 01:49:54.368814774 +0000 UTC m=+0.543822545 container remove 7c46298081691f565ee2168ff505e1138998b760e8729cd244c09fb6d6c51894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424) 2024-04-25T01:49:55.044 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:54 smithi196 podman[68354]: 2024-04-25 01:49:54.670035412 +0000 UTC m=+0.017560339 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:49:55.044 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:54 smithi196 podman[68354]: 2024-04-25 01:49:54.794731528 +0000 UTC m=+0.142256448 container create dd0ad29378ad69f5bfd35c0e8a91da9dbdfee64f83027a2514cfa290666f5446 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T01:49:55.044 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:55 smithi196 podman[68354]: 2024-04-25 01:49:55.046964927 +0000 UTC m=+0.394489847 container init dd0ad29378ad69f5bfd35c0e8a91da9dbdfee64f83027a2514cfa290666f5446 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T01:49:55.340 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:55 smithi196 podman[68354]: 2024-04-25 01:49:55.04997256 +0000 UTC m=+0.397497483 container start dd0ad29378ad69f5bfd35c0e8a91da9dbdfee64f83027a2514cfa290666f5446 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T01:49:55.340 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:55 smithi196 podman[68354]: 2024-04-25 01:49:55.13025275 +0000 UTC m=+0.477777688 container attach dd0ad29378ad69f5bfd35c0e8a91da9dbdfee64f83027a2514cfa290666f5446 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T01:49:55.622 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 25 01:49:55 smithi196 podman[68354]: 2024-04-25 01:49:55.343547985 +0000 UTC m=+0.691072906 container died dd0ad29378ad69f5bfd35c0e8a91da9dbdfee64f83027a2514cfa290666f5446 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T01:49:55.916 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T01:49:55.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:49:55.955 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T01:49:55.955 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T01:49:55.955 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.a 2024-04-25T01:49:56.246 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:49:55 smithi078 systemd[1]: Stopping Ceph mds.a for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:49:56.512 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:49:56 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-a[68467]: 2024-04-25T01:49:56.248+0000 7f16f41c9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:49:56.513 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:49:56 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-a[68467]: 2024-04-25T01:49:56.248+0000 7f16f41c9700 -1 mds.a *** got signal Terminated *** 2024-04-25T01:49:59.513 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:49:59 smithi078 podman[84993]: 2024-04-25 01:49:59.263376457 +0000 UTC m=+3.120097850 container died b1e800d9d8d9faa251a79571bd4c857502958ffc28cc7f79ae3a123a873dadd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T01:50:00.005 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:49:59 smithi078 podman[84993]: 2024-04-25 01:49:59.815221425 +0000 UTC m=+3.671942816 container cleanup b1e800d9d8d9faa251a79571bd4c857502958ffc28cc7f79ae3a123a873dadd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-25T01:50:00.005 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:49:59 smithi078 bash[84993]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-a 2024-04-25T01:50:00.237 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T01:50:00.257 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:50:00 smithi078 podman[85007]: 2024-04-25 01:50:00.008277459 +0000 UTC m=+0.737348449 container remove b1e800d9d8d9faa251a79571bd4c857502958ffc28cc7f79ae3a123a873dadd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:50:00.257 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:50:00 smithi078 systemd[1]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.a.service: Deactivated successfully. 2024-04-25T01:50:00.257 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:50:00 smithi078 systemd[1]: Stopped Ceph mds.a for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:50:00.257 INFO:journalctl@ceph.mds.a.smithi078.stdout:Apr 25 01:50:00 smithi078 systemd[1]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.a.service: Consumed 1.320s CPU time. 2024-04-25T01:50:00.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:50:00.270 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T01:50:00.270 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T01:50:00.270 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.d 2024-04-25T01:50:00.513 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:50:00 smithi078 systemd[1]: Stopping Ceph mds.d for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:50:01.013 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:50:00 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-d[67507]: 2024-04-25T01:50:00.611+0000 7f92b105a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:50:01.013 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:50:00 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-d[67507]: 2024-04-25T01:50:00.611+0000 7f92b105a700 -1 mds.d *** got signal Terminated *** 2024-04-25T01:50:04.893 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:50:04 smithi078 podman[85114]: 2024-04-25 01:50:04.625396427 +0000 UTC m=+4.138199418 container died 81055bfaff020ed170910cafc700c8dd59a9d1f044b0f803f4a8f88d4debff44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T01:50:05.260 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:50:05 smithi078 podman[85114]: 2024-04-25 01:50:05.061216394 +0000 UTC m=+4.574019387 container cleanup 81055bfaff020ed170910cafc700c8dd59a9d1f044b0f803f4a8f88d4debff44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T01:50:05.260 INFO:journalctl@ceph.mds.d.smithi078.stdout:Apr 25 01:50:05 smithi078 bash[85114]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-d 2024-04-25T01:50:05.479 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T01:50:05.512 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:50:05.512 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T01:50:05.512 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T01:50:05.512 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.g 2024-04-25T01:50:05.763 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:50:05 smithi078 systemd[1]: Stopping Ceph mds.g for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:50:06.263 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:50:05 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-g[68108]: 2024-04-25T01:50:05.865+0000 7f479e83b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:50:06.263 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:50:05 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-g[68108]: 2024-04-25T01:50:05.865+0000 7f479e83b700 -1 mds.g *** got signal Terminated *** 2024-04-25T01:50:07.513 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:50:07 smithi078 podman[85236]: 2024-04-25 01:50:07.259852938 +0000 UTC m=+1.529182083 container died e27d76e841e0f09dfc17160d2fe938339767fbdecd02f0eb2d6e91caf7865e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:50:07.789 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:50:07 smithi078 podman[85236]: 2024-04-25 01:50:07.5988867 +0000 UTC m=+1.868215833 container cleanup e27d76e841e0f09dfc17160d2fe938339767fbdecd02f0eb2d6e91caf7865e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T01:50:07.789 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:50:07 smithi078 bash[85236]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-g 2024-04-25T01:50:08.021 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T01:50:08.041 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:50:07 smithi078 podman[85251]: 2024-04-25 01:50:07.791801752 +0000 UTC m=+0.525426968 container remove e27d76e841e0f09dfc17160d2fe938339767fbdecd02f0eb2d6e91caf7865e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-g, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T01:50:08.041 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:50:08 smithi078 systemd[1]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.g.service: Deactivated successfully. 2024-04-25T01:50:08.041 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:50:08 smithi078 systemd[1]: Stopped Ceph mds.g for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:50:08.041 INFO:journalctl@ceph.mds.g.smithi078.stdout:Apr 25 01:50:08 smithi078 systemd[1]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.g.service: Consumed 1.268s CPU time. 2024-04-25T01:50:08.052 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:50:08.052 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T01:50:08.052 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T01:50:08.052 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.j 2024-04-25T01:50:08.375 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:50:08 smithi078 systemd[1]: Stopping Ceph mds.j for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:50:08.763 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:50:08 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-j[67154]: 2024-04-25T01:50:08.377+0000 7f9cbc4a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:50:08.763 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:50:08 smithi078 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-j[67154]: 2024-04-25T01:50:08.377+0000 7f9cbc4a4700 -1 mds.j *** got signal Terminated *** 2024-04-25T01:50:10.263 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:50:09 smithi078 podman[85356]: 2024-04-25 01:50:09.983661813 +0000 UTC m=+1.715776743 container died 9a1216143646353ca9b984db4adea23861975affd340e8b309c488837962100f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:50:10.763 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:50:10 smithi078 podman[85356]: 2024-04-25 01:50:10.428302386 +0000 UTC m=+2.160417352 container cleanup 9a1216143646353ca9b984db4adea23861975affd340e8b309c488837962100f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T01:50:10.763 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:50:10 smithi078 bash[85356]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-j 2024-04-25T01:50:10.763 INFO:journalctl@ceph.mds.j.smithi078.stdout:Apr 25 01:50:10 smithi078 podman[85369]: 2024-04-25 01:50:10.604337665 +0000 UTC m=+0.619382174 container remove 9a1216143646353ca9b984db4adea23861975affd340e8b309c488837962100f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:50:10.847 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T01:50:10.877 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:50:10.878 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T01:50:10.878 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T01:50:10.878 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.b 2024-04-25T01:50:11.305 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:50:10 smithi152 systemd[1]: Stopping Ceph mds.b for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:50:11.305 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:50:11 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-b[62804]: 2024-04-25T01:50:11.149+0000 7f215355f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:50:11.306 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:50:11 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-b[62804]: 2024-04-25T01:50:11.149+0000 7f215355f700 -1 mds.b *** got signal Terminated *** 2024-04-25T01:50:15.457 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:50:15 smithi152 podman[66863]: 2024-04-25 01:50:15.162811185 +0000 UTC m=+4.094265255 container died 87728433819dba640e7aa2b4a417f3e4d35901504b41035a5489e72442da21f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T01:50:15.798 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:50:15 smithi152 podman[66863]: 2024-04-25 01:50:15.550080358 +0000 UTC m=+4.481534414 container cleanup 87728433819dba640e7aa2b4a417f3e4d35901504b41035a5489e72442da21f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T01:50:15.798 INFO:journalctl@ceph.mds.b.smithi152.stdout:Apr 25 01:50:15 smithi152 bash[66863]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-b 2024-04-25T01:50:15.931 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T01:50:15.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:50:15.967 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T01:50:15.967 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T01:50:15.967 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.e 2024-04-25T01:50:16.054 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:50:16 smithi152 systemd[1]: Stopping Ceph mds.e for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:50:16.805 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:50:16 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-e[62178]: 2024-04-25T01:50:16.345+0000 7fa3658a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:50:16.805 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:50:16 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-e[62178]: 2024-04-25T01:50:16.345+0000 7fa3658a0700 -1 mds.e *** got signal Terminated *** 2024-04-25T01:50:19.555 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:50:19 smithi152 podman[66997]: 2024-04-25 01:50:19.265035709 +0000 UTC m=+3.082378288 container died ebc5baaea7ef0aa1615997a1f13bcfb8c1a01e7b0ec0e1decdb18cd4b66cccde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T01:50:19.886 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:50:19 smithi152 podman[66997]: 2024-04-25 01:50:19.654215496 +0000 UTC m=+3.471558072 container cleanup ebc5baaea7ef0aa1615997a1f13bcfb8c1a01e7b0ec0e1decdb18cd4b66cccde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:50:19.886 INFO:journalctl@ceph.mds.e.smithi152.stdout:Apr 25 01:50:19 smithi152 bash[66997]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-e 2024-04-25T01:50:20.030 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T01:50:20.062 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:50:20.063 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T01:50:20.063 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T01:50:20.063 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.h 2024-04-25T01:50:20.389 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:50:20 smithi152 systemd[1]: Stopping Ceph mds.h for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:50:20.805 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:50:20 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-h[62501]: 2024-04-25T01:50:20.391+0000 7f5818289700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:50:20.805 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:50:20 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-h[62501]: 2024-04-25T01:50:20.391+0000 7f5818289700 -1 mds.h *** got signal Terminated *** 2024-04-25T01:50:23.539 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:50:23 smithi152 podman[67119]: 2024-04-25 01:50:23.278517182 +0000 UTC m=+3.000462261 container died b6b3e0c9d0eef445ef3cbe466aa67c34fa749b5e849c394a1cacce7b507a1ef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-25T01:50:23.805 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:50:23 smithi152 podman[67119]: 2024-04-25 01:50:23.632994672 +0000 UTC m=+3.354939756 container cleanup b6b3e0c9d0eef445ef3cbe466aa67c34fa749b5e849c394a1cacce7b507a1ef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T01:50:23.806 INFO:journalctl@ceph.mds.h.smithi152.stdout:Apr 25 01:50:23 smithi152 bash[67119]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-h 2024-04-25T01:50:24.054 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T01:50:24.084 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:50:24.085 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T01:50:24.085 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T01:50:24.085 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.k 2024-04-25T01:50:24.426 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:50:24 smithi152 systemd[1]: Stopping Ceph mds.k for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:50:24.805 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:50:24 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-k[61860]: 2024-04-25T01:50:24.428+0000 7f6fe0200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:50:24.805 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:50:24 smithi152 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-k[61860]: 2024-04-25T01:50:24.428+0000 7f6fe0200700 -1 mds.k *** got signal Terminated *** 2024-04-25T01:50:27.518 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:50:27 smithi152 podman[67240]: 2024-04-25 01:50:27.26677366 +0000 UTC m=+2.955169014 container died 7d32f90d704370927879f6d7048b12a514624219a3bb49ca8ce11942fcf1970a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T01:50:27.768 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:50:27 smithi152 podman[67240]: 2024-04-25 01:50:27.586830662 +0000 UTC m=+3.275226017 container cleanup 7d32f90d704370927879f6d7048b12a514624219a3bb49ca8ce11942fcf1970a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T01:50:27.769 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:50:27 smithi152 bash[67240]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-k 2024-04-25T01:50:28.003 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T01:50:28.022 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:50:27 smithi152 podman[67254]: 2024-04-25 01:50:27.771194673 +0000 UTC m=+0.501628924 container remove 7d32f90d704370927879f6d7048b12a514624219a3bb49ca8ce11942fcf1970a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T01:50:28.022 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:50:28 smithi152 systemd[1]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-25T01:50:28.022 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:50:28 smithi152 systemd[1]: Stopped Ceph mds.k for df36bea8-02a3-11ef-bc93-c7b262605968. 2024-04-25T01:50:28.023 INFO:journalctl@ceph.mds.k.smithi152.stdout:Apr 25 01:50:28 smithi152 systemd[1]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.k.service: Consumed 1.389s CPU time. 2024-04-25T01:50:28.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:50:28.033 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T01:50:28.033 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T01:50:28.033 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.c 2024-04-25T01:50:28.373 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:50:28 smithi196 systemd[1]: Stopping Ceph mds.c for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:50:28.634 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:50:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-c[63966]: 2024-04-25T01:50:28.415+0000 7f0c0bfea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:50:28.634 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:50:28 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-c[63966]: 2024-04-25T01:50:28.415+0000 7f0c0bfea700 -1 mds.c *** got signal Terminated *** 2024-04-25T01:50:31.623 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:50:31 smithi196 podman[68462]: 2024-04-25 01:50:31.281918638 +0000 UTC m=+3.046367280 container died 35beadf15ca7257a69719533a0b321384fb6d616a5e6b4cbef7d5e6802b5943d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:50:32.038 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:50:31 smithi196 podman[68462]: 2024-04-25 01:50:31.731631266 +0000 UTC m=+3.496079906 container cleanup 35beadf15ca7257a69719533a0b321384fb6d616a5e6b4cbef7d5e6802b5943d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T01:50:32.039 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 25 01:50:31 smithi196 bash[68462]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-c 2024-04-25T01:50:32.204 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T01:50:32.239 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:50:32.239 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T01:50:32.239 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T01:50:32.239 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.f 2024-04-25T01:50:32.623 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:50:32 smithi196 systemd[1]: Stopping Ceph mds.f for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:50:32.623 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:50:32 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-f[63658]: 2024-04-25T01:50:32.544+0000 7f8777ee7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:50:32.623 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:50:32 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-f[63658]: 2024-04-25T01:50:32.544+0000 7f8777ee7700 -1 mds.f *** got signal Terminated *** 2024-04-25T01:50:35.542 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:50:35 smithi196 podman[68596]: 2024-04-25 01:50:35.280858169 +0000 UTC m=+2.826258591 container died 0bf5b3311815aa67433665a76f09360a6ff9eba037a94378c3cd45e5264cf5f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T01:50:35.826 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:50:35 smithi196 podman[68596]: 2024-04-25 01:50:35.627375636 +0000 UTC m=+3.172776068 container cleanup 0bf5b3311815aa67433665a76f09360a6ff9eba037a94378c3cd45e5264cf5f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-25T01:50:35.826 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 25 01:50:35 smithi196 bash[68596]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-f 2024-04-25T01:50:36.050 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T01:50:36.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:50:36.082 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T01:50:36.082 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T01:50:36.082 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.i 2024-04-25T01:50:36.373 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:50:36 smithi196 systemd[1]: Stopping Ceph mds.i for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:50:36.873 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:50:36 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-i[64283]: 2024-04-25T01:50:36.397+0000 7f3b547bd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:50:36.873 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:50:36 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-i[64283]: 2024-04-25T01:50:36.397+0000 7f3b547bd700 -1 mds.i *** got signal Terminated *** 2024-04-25T01:50:39.373 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:50:39 smithi196 podman[68716]: 2024-04-25 01:50:39.279703631 +0000 UTC m=+2.985701736 container died 3241e6bf050a2c9d4a62fb33aaf745011eb38ec7c5d88386da4160a70d568a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:50:39.873 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:50:39 smithi196 podman[68716]: 2024-04-25 01:50:39.714727029 +0000 UTC m=+3.420725132 container cleanup 3241e6bf050a2c9d4a62fb33aaf745011eb38ec7c5d88386da4160a70d568a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T01:50:39.873 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 25 01:50:39 smithi196 bash[68716]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-i 2024-04-25T01:50:40.131 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T01:50:40.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:50:40.163 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T01:50:40.163 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T01:50:40.164 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.l 2024-04-25T01:50:40.474 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:50:40 smithi196 systemd[1]: Stopping Ceph mds.l for df36bea8-02a3-11ef-bc93-c7b262605968... 2024-04-25T01:50:40.873 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:50:40 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-l[63352]: 2024-04-25T01:50:40.476+0000 7f07f96ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:50:40.873 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:50:40 smithi196 ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-l[63352]: 2024-04-25T01:50:40.476+0000 7f07f96ae700 -1 mds.l *** got signal Terminated *** 2024-04-25T01:50:43.623 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:50:43 smithi196 podman[68838]: 2024-04-25 01:50:43.27407692 +0000 UTC m=+2.896830282 container died e124f5eb5ddfe61e28a42a8e83f9b639314ccdd48efb9e8c68ef7c6b70aa7c9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9) 2024-04-25T01:50:43.916 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:50:43 smithi196 podman[68838]: 2024-04-25 01:50:43.709363964 +0000 UTC m=+3.332117323 container cleanup e124f5eb5ddfe61e28a42a8e83f9b639314ccdd48efb9e8c68ef7c6b70aa7c9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-25T01:50:43.916 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 25 01:50:43 smithi196 bash[68838]: ceph-df36bea8-02a3-11ef-bc93-c7b262605968-mds-l 2024-04-25T01:50:44.132 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df36bea8-02a3-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T01:50:44.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T01:50:44.162 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T01:50:44.163 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid df36bea8-02a3-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T01:50:44.348 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:50:46.739 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid df36bea8-02a3-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T01:50:46.931 INFO:teuthology.orchestra.run.smithi152.stdout:Deleting cluster with fsid: df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:50:49.388 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid df36bea8-02a3-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T01:50:49.576 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:50:51.837 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:50:51.868 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:50:51.900 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:50:51.927 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T01:50:51.929 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329/remote/smithi078/crash 2024-04-25T01:50:51.930 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/crash -- . 2024-04-25T01:50:51.970 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T01:50:51.970 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now 2024-04-25T01:50:51.972 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329/remote/smithi152/crash 2024-04-25T01:50:51.972 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/crash -- . 2024-04-25T01:50:52.017 INFO:teuthology.orchestra.run.smithi152.stderr:tar: /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T01:50:52.017 INFO:teuthology.orchestra.run.smithi152.stderr:tar: Error is not recoverable: exiting now 2024-04-25T01:50:52.018 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329/remote/smithi196/crash 2024-04-25T01:50:52.019 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/crash -- . 2024-04-25T01:50:52.079 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T01:50:52.079 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-04-25T01:50:52.081 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T01:50:52.081 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T01:50:52.118 INFO:tasks.cephadm:Compressing logs... 2024-04-25T01:50:52.118 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T01:50:52.161 DEBUG:teuthology.orchestra.run.smithi152:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T01:50:52.164 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T01:50:52.185 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T01:50:52.185 INFO:teuthology.orchestra.run.smithi078.stderr:: No such file or directory 2024-04-25T01:50:52.185 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-25T01:50:52.185 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.log 2024-04-25T01:50:52.185 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T01:50:52.185 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T01:50:52.191 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T01:50:52.191 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T01:50:52.191 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.j.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-25T01:50:52.191 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T01:50:52.191 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T01:50:52.191 INFO:teuthology.orchestra.run.smithi078.stderr: 86.1%/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.0.log: -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T01:50:52.191 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T01:50:52.191 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T01:50:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mon.a.log: 90.0% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T01:50:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T01:50:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.6.log: 94.8% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T01:50:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T01:50:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.0% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T01:50:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T01:50:52.194 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.a.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T01:50:52.202 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T01:50:52.202 INFO:teuthology.orchestra.run.smithi196.stderr:: No such file or directory 2024-04-25T01:50:52.202 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.log 2024-04-25T01:50:52.202 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T01:50:52.202 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T01:50:52.202 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T01:50:52.202 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T01:50:52.202 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T01:50:52.202 INFO:teuthology.orchestra.run.smithi196.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T01:50:52.202 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T01:50:52.203 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T01:50:52.203 INFO:teuthology.orchestra.run.smithi196.stderr: 86.1%/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mon.c.log: -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T01:50:52.203 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T01:50:52.203 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T01:50:52.203 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.2% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T01:50:52.203 INFO:teuthology.orchestra.run.smithi196.stderr: 92.3% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T01:50:52.205 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T01:50:52.206 INFO:teuthology.orchestra.run.smithi152.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T01:50:52.206 INFO:teuthology.orchestra.run.smithi152.stderr:: No such file or directory 2024-04-25T01:50:52.207 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.log 2024-04-25T01:50:52.207 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T01:50:52.207 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T01:50:52.207 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T01:50:52.207 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T01:50:52.207 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.7.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T01:50:52.207 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T01:50:52.207 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T01:50:52.207 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose 86.0% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T01:50:52.207 INFO:teuthology.orchestra.run.smithi152.stderr: -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T01:50:52.207 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T01:50:52.207 INFO:teuthology.orchestra.run.smithi152.stderr: 90.1% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T01:50:52.207 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T01:50:52.207 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T01:50:52.208 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.1% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T01:50:52.208 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T01:50:52.208 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.e.log: 92.1% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T01:50:52.208 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T01:50:52.209 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T01:50:52.210 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T01:50:52.214 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T01:50:52.214 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T01:50:52.214 INFO:teuthology.orchestra.run.smithi196.stderr: 82.9% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T01:50:52.214 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T01:50:52.215 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-volume.log: 93.8% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T01:50:52.215 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T01:50:52.216 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T01:50:52.218 INFO:teuthology.orchestra.run.smithi078.stderr: 94.6% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T01:50:52.225 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.8.log: 93.9% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T01:50:52.226 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T01:50:52.226 INFO:teuthology.orchestra.run.smithi078.stderr: 93.8% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T01:50:52.232 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.k.log: 93.9% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T01:50:52.240 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T01:50:52.243 INFO:teuthology.orchestra.run.smithi196.stderr: 93.4% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T01:50:52.247 INFO:teuthology.orchestra.run.smithi152.stderr: 93.3% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T01:50:52.278 INFO:teuthology.orchestra.run.smithi152.stderr: 93.4% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T01:50:52.304 INFO:teuthology.orchestra.run.smithi196.stderr: 93.6% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T01:50:52.392 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.9.log: 89.8% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T01:50:52.461 INFO:teuthology.orchestra.run.smithi196.stderr: 91.9% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T01:50:52.488 INFO:teuthology.orchestra.run.smithi152.stderr: 92.0% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T01:50:52.619 INFO:teuthology.orchestra.run.smithi196.stderr: 95.8% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T01:50:52.709 INFO:teuthology.orchestra.run.smithi078.stderr: 90.7% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T01:50:53.287 INFO:teuthology.orchestra.run.smithi196.stderr: 92.0% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T01:50:53.475 INFO:teuthology.orchestra.run.smithi078.stderr: 92.0% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T01:50:53.732 INFO:teuthology.orchestra.run.smithi078.stderr: 92.0% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T01:50:53.862 INFO:teuthology.orchestra.run.smithi152.stderr: 92.3% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T01:50:53.952 INFO:teuthology.orchestra.run.smithi152.stderr: 92.3% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T01:50:54.007 INFO:teuthology.orchestra.run.smithi196.stderr: 92.2% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T01:50:54.066 INFO:teuthology.orchestra.run.smithi152.stderr: 92.2% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T01:50:54.122 INFO:teuthology.orchestra.run.smithi078.stderr: 92.4% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T01:50:54.337 INFO:teuthology.orchestra.run.smithi196.stderr: 93.5% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T01:50:54.507 INFO:teuthology.orchestra.run.smithi078.stderr: 92.8% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T01:50:54.509 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-04-25T01:50:54.509 INFO:teuthology.orchestra.run.smithi078.stderr:real 0m2.335s 2024-04-25T01:50:54.509 INFO:teuthology.orchestra.run.smithi078.stderr:user 0m7.618s 2024-04-25T01:50:54.509 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m0.269s 2024-04-25T01:50:54.849 INFO:teuthology.orchestra.run.smithi152.stderr: 92.6% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T01:50:56.150 INFO:teuthology.orchestra.run.smithi152.stderr: 93.5% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T01:50:56.152 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-04-25T01:50:56.152 INFO:teuthology.orchestra.run.smithi152.stderr:real 0m3.974s 2024-04-25T01:50:56.152 INFO:teuthology.orchestra.run.smithi152.stderr:user 0m11.981s 2024-04-25T01:50:56.152 INFO:teuthology.orchestra.run.smithi152.stderr:sys 0m0.399s 2024-04-25T01:50:57.408 INFO:teuthology.orchestra.run.smithi196.stderr: 94.9% -- replaced with /var/log/ceph/df36bea8-02a3-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T01:50:57.409 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-04-25T01:50:57.409 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m5.231s 2024-04-25T01:50:57.410 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m10.732s 2024-04-25T01:50:57.410 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.366s 2024-04-25T01:50:57.410 INFO:tasks.cephadm:Archiving logs... 2024-04-25T01:50:57.410 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329/remote/smithi078/log 2024-04-25T01:50:57.411 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:50:58.694 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329/remote/smithi152/log 2024-04-25T01:50:58.695 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:51:00.493 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329/remote/smithi196/log 2024-04-25T01:51:00.493 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:51:02.084 INFO:tasks.cephadm:Removing cluster... 2024-04-25T01:51:02.084 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid df36bea8-02a3-11ef-bc93-c7b262605968 --force 2024-04-25T01:51:02.280 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:51:02.542 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid df36bea8-02a3-11ef-bc93-c7b262605968 --force 2024-04-25T01:51:02.734 INFO:teuthology.orchestra.run.smithi152.stdout:Deleting cluster with fsid: df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:51:02.997 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid df36bea8-02a3-11ef-bc93-c7b262605968 --force 2024-04-25T01:51:03.177 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: df36bea8-02a3-11ef-bc93-c7b262605968 2024-04-25T01:51:03.430 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T01:51:03.430 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T01:51:03.447 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T01:51:03.464 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T01:51:03.479 INFO:tasks.cephadm:Teardown complete 2024-04-25T01:51:03.480 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T01:51:03.493 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T01:51:03.494 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:51:03.497 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:51:03.507 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:51:03.618 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T01:51:03.635 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T01:51:03.636 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:51:03.640 DEBUG:teuthology.orchestra.run.smithi152:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:51:03.642 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:51:03.652 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:51:03.654 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:51:03.656 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:51:03.656 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:51:03.657 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-04-25T01:51:03.657 INFO:teuthology.orchestra.run.smithi078.stdout:^- hv01.front.sepia.ceph.com 2 7 377 50 +1630us[+1635us] +/- 567ms 2024-04-25T01:51:03.657 INFO:teuthology.orchestra.run.smithi078.stdout:^* hv02.front.sepia.ceph.com 3 7 377 49 -208us[ -204us] +/- 71ms 2024-04-25T01:51:03.657 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 53 +299us[ +303us] +/- 78ms 2024-04-25T01:51:03.657 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:51:03.659 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:51:03.659 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-04-25T01:51:03.659 INFO:teuthology.orchestra.run.smithi152.stdout:^- hv01.front.sepia.ceph.com 2 6 377 51 +1585us[+1585us] +/- 567ms 2024-04-25T01:51:03.659 INFO:teuthology.orchestra.run.smithi152.stdout:^* hv02.front.sepia.ceph.com 3 6 377 52 -236us[ -237us] +/- 71ms 2024-04-25T01:51:03.659 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +311us[ +311us] +/- 78ms 2024-04-25T01:51:03.659 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:51:03.661 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:51:03.661 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-04-25T01:51:03.661 INFO:teuthology.orchestra.run.smithi196.stdout:^- hv01.front.sepia.ceph.com 2 6 377 54 +1605us[+1609us] +/- 567ms 2024-04-25T01:51:03.661 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv02.front.sepia.ceph.com 3 6 377 53 -250us[ -247us] +/- 71ms 2024-04-25T01:51:03.661 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 +285us[ +285us] +/- 78ms 2024-04-25T01:51:03.661 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:51:03.661 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T01:51:03.675 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T01:51:03.676 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T01:51:03.687 DEBUG:teuthology.orchestra.run.smithi078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T01:51:03.771 DEBUG:teuthology.orchestra.run.smithi152:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T01:51:03.835 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T01:51:03.897 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:51:03.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:51:03.926 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:51:03.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:51:03.952 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:51:03.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:51:03.979 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T01:51:04.005 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T01:51:04.015 INFO:teuthology.task.internal:Duration was 1403.729512 seconds 2024-04-25T01:51:04.015 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T01:51:04.025 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T01:51:04.025 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:51:04.028 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:51:04.030 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:51:04.062 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:51:04.064 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:51:04.066 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:51:04.340 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T01:51:04.340 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2024-04-25T01:51:04.341 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:51:04.368 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi152.front.sepia.ceph.com 2024-04-25T01:51:04.368 DEBUG:teuthology.orchestra.run.smithi152:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:51:04.407 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T01:51:04.408 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:51:04.436 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T01:51:04.436 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:51:04.439 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:51:04.453 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:51:04.502 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T01:51:04.502 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:51:04.505 DEBUG:teuthology.orchestra.run.smithi152:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:51:04.519 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:51:04.876 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T01:51:04.891 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T01:51:04.892 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:51:04.915 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:51:04.940 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:51:04.964 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T01:51:04.984 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:51:04.987 DEBUG:teuthology.orchestra.run.smithi152:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:51:04.990 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:51:05.006 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2024-04-25T01:51:05.009 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = core 2024-04-25T01:51:05.027 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-04-25T01:51:05.056 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:51:05.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:51:05.073 DEBUG:teuthology.orchestra.run.smithi152:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:51:05.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:51:05.087 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:51:05.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:51:05.112 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T01:51:05.123 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T01:51:05.124 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329/remote/smithi078 2024-04-25T01:51:05.124 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:51:05.164 DEBUG:teuthology.misc:Transferring archived files from smithi152:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329/remote/smithi152 2024-04-25T01:51:05.165 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:51:05.199 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672329/remote/smithi196 2024-04-25T01:51:05.200 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:51:05.235 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T01:51:05.235 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:51:05.237 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:51:05.239 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:51:05.284 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T01:51:05.296 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T01:51:05.296 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T01:51:05.307 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T01:51:05.307 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:51:05.309 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:51:05.311 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:51:05.321 INFO:teuthology.orchestra.run.smithi078.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:51 /home/ubuntu/cephtest 2024-04-25T01:51:05.323 INFO:teuthology.orchestra.run.smithi152.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:51 /home/ubuntu/cephtest 2024-04-25T01:51:05.338 INFO:teuthology.orchestra.run.smithi196.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:51 /home/ubuntu/cephtest 2024-04-25T01:51:05.339 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T01:51:05.350 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T01:51:05.413 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/direct_io}} duration: 1403.7295124530792 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T01:51:05.414 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T01:51:05.497 INFO:teuthology.run:pass